March  2014 


Education  and  Professional  Development  Colloquium  2014 


Volume  47v'  NurnB 

In  This  Issue:  il  l  5  li 


series 


Ln 


Report  Documentation  Page 

Form  Approved 

0MB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  0MB  control  number. 

1.  REPORT  DATE 

MAR  2014  TYPE 

3.  DATES  COVERED 

00-00-2014  to  00-00-2014 

4.  TITLE  AND  SUBTITLE 

Phalanx.  Volume  47,  Number  1. 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Military  Operations  Research  Society ,2111  Wilson  Boulevard,  Suite 
700,Arlington,VA,22201 

8.  PEREORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIEICATION  OE:  17.  LIMITATION  OE 

ARSTRAUT 

18.  NUMBER  19a.  NAME  OE 

OE  PAGES  RESPONSIBLE  PERSON 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE  Sume  US 

unclassified  unclassified  unclassified  Report  (SAR) 

56 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


VISUAL 

ANALYTICS 

See  your  data  for  all  it's  worth. 

Explore  billions  of  rows  of  data  in  minutes  or  seconds,  visually  represented  in  a  way  that  brings  hidden  patterns 
into  plain  sight.  Then  easily  create  and  share  reports  on  the  Web  or  mobile  devices. 


sas.com/vadata 

Experience  Visual  Analytics  firsthand. 


S.sas 

THE  POWER  TO  KNOW® 


SAS  Institute  Inc.  product  or  service  names  ore  registered  trademarks  or  trademarks  of  SAS  Institute  Inc.  in  the  USA  and  other  countries. 


'  indicates  USA  registration.  Other  brand  and  product  names  are  trademarks  of  their  respective  companies.  ©  2013  SAS  Institute  Inc.  All  rights  reserved.  S1058220US.0313 


Military  Operations  Research  Society 
21 1 1  Wilson  Blvd. 

Suite  700 

Arlington,  VA  22201 
703  933.9070;  FAX  703  933.9066 
e-mail:  morsoffice@mors.org 
www.mors.org 


MORS/MAS 


MORS  OFFICERS 


. MORS  President 

.Sustaining  Contributors 
. MAS  President 


PRESIDENT 

Dr.  Stephen  R.  Riese  (Steve) 
Johns  Hopkins  University 
Applied  Physics  Lab 
stephen.riese@jhuapl.edu 

PRESIDENT  ELECT 

Dr.  Rafael  E.  Matos  (Raf) 

WBB  Consulting,  Inc. 
MatosMORS@gmail.com 

VICE  PRESIDENT 
(FINANCE  AND  MANAGEMENT) 

Dr.  Donna  W.  Blake 
PEMDAS  Technologies 
Donnawblake@verison.net 

VICE  PRESIDENT 
(MEETING  OPERATIONS) 

Tom  Denesia 

NORAD-USNORTHCOM/J84 

Thomas.denesia@northcom.mil 


EPD  ColloquiuiiPi 

'  m 

...Webinar  Series 


JwK . Industry  Day 

lll^  — . Affordability  Analysis 

. . Test  an'd  'AnaJvsisTech^  Trainlno  Workshop 

K... . . Updat^n  the  82nd MORS  Symp^ium 

Katrina  G.  McFarlan'd,  As^ta^  S^re^r^f  Defense  (Acquisition) 


JURES 


VICE  PRESIDENT 
(MEMBER  &  SOCIETAL  SERVICES) 

Dr.  Julie  A.  Seton 
Indelible  Enterprises  LLC 
Julie@indentus.com 

—  SECRETARY  OF  THE  SOCIETY 

Robert  R.  Koury 
Price  Systems 

bob.koury@pricesystems.com 

IMMEDIATE  PAST  PRESIDENT 

Michael  W.  Garrambone 
InfoSciTex 

mgarrambone@aol.com 

CHIEF  EXECUTIVE  OFFICER 

Susan  K.  Reardon 

Military  Operations  Research  Society 
susan@mors.org 

The  Phalanx  (ISSN  0195-1 920)  Is  published  quarterly, 
$40.00  for  one  year  or  $70.00  for  two  years  (US 
Rates)  by  the  Military  Operations  Research  Society  in 
cooperation  with  the  Military  Applications'^oclefy 
(MAS)  of  the  Institute  for  Operations  Research  and 
Management  Science  (INFORMS).  Principal  office: 
2111  Wilson  Blvd,  Suite  700,  Arlington,  Virginia  22201. 
Periodicals  postage  paid  at  Arlington,  and  at  ad¬ 
ditional  mailing  offices.  POSTMASTER:  Send  address 
changes  to  Phalanx,  2111  Wilson  Blvd,  Suite  700, 
Arlington,  Virginia  22201.  Please  allow  4-6  weeks  for 
address  change  activation. 

INFORMS 

5521  Research  Park  Drive,  Suite  200, 
Catonsville,  MD  21228 
800-4INFORMS  FAX  443.757.351 5 
e-mail:  informs@informs.org 

MAS  COUNCIL 

EXECUTIVE  COMMITTEE 
WILLIAM  P.  FOX 
President 
GREG  H.  PARLIER 
Past  President 


Allied  M®h( 


Action  G 


ison 


TARY 


Corner:  Bi 


INTOTgl 


•^Cornwallis  Jom  Forces 

.P.f. . MORSi^tag^ 

ri:r  | 
^Jnbute:  Sy  Deitcjnmari 

. . PuzzlOR 

M.r...  Member  l^leltones' 

1  j.,  ■ 


LAST  WORD 
50 . 


Guest  Editorial  -  Arthur  H.  Barber  III 


Phalanx  STAFF 


Ec//[pr; Terry  McKearney,The  Ranger  Group, 
t^ry.mckearn^@therangergroup.com 
Prodi}^j^  Editor,  Joan  Taylor,  MORS, 
joarf@mors.org 

Graphic  Design/Layout:  Mike  Noonan 


Num^0l^  from  Operations,  George  W.S.  Kuhn, 
.gkuhn@imi.org 
The  Pseudo-Analytical  Agenda, 

Wright  Handsides 


Cindy  Grier,  TRAC-FLVN 

Tim  Hope,  WBB  Consulting 

Douglas  E.  Otte,  Naval  Postgraduate  School 

Deborah  Ray,  US  Army  Manpower  Analysis  Agency 

Julie  Seton,  Indelible  Enterprises,  LLC 

Bob  Sheldon,  FS,  Group  W.,  Inc. 

COL  Steven  Stoddard,  Center  for  Army  Analysis 


Phalanx  Editorial  Board 

ffnair: Greg  H.  Parlier,  IDA 


Department  Editors 

Naval  Anal-^,  Brian  G.  McCue,CNA, 
brianmccL^olum.mit.edu  ■ 

Letters  to  the  Editor,  MORS  Office, 
morsoffice@rr^.org 

Modeling  and  Simulation,  James  N.  Bexfield,  FS, 
jim_bexfteld@comcast.net 
MOR  Eleritage,  Eugene  RVisco,  FS, 
genevisco@embarqmail.com 


COUNCIL  MEMBERS 

MAHYAR  AMOUZEGAR 
CHRIS  ARNEY 
JEFF  EATON 
CHAD  LONG 
DOUG  MATTY 
GEORGE  MAYERNIK 


MORS  Publications  Committee 

Dr.  Donna  Blake,  PEMDAS  Technologies 
&  Innovations 

Dr.  Richard  Deckro,  FS,  AFIT 
Dr.  Jerry  Diaz,  ASAF,  A1 PF 
Brian  Engler,  FS 

Dr.  Niki  Goerger,  USACE/ERDC  LNO  to  ASA(ALT) 


Subscriptions  to  Phalanx  are  included  In  the  annual 
dues  of  both  INF0RMS7MAS  and  MORS  members. 


DISCIAiMER:  MORS  and  MAS  are  professional  societies  for  people,  not  organizations.  As  such,  the  articles  published  in  Phalanx  represent  the  professional 
views  qndjudgments  of  individuals  independent  of  the  organization  that  may  employ  them  or  contract  for  their  services.  Except  where  specifically  identi¬ 
fied,  kotements  and  opinions  do  not  necessarily  reflect  policies  or  positions  of  the  Department  of  Defense  or  any  other  agency  of  the  US  Government 


Printed 
On  Recycled 


L 

Paper  ©2013,  Military  Op^ra^ns  Research  Society  and  Military  Application  Society. 


MORS/MAS 


An  Adaptive 

Supporting  Your  Career 

Dr.  Stephen  R.  Riese,  MORS  President,  stephen.riese@jhuapl.edu 


tions,  I  am  happy  to  report  that 
MORS  is  indeed  iearning,  adapting, 
and  surviving,  and  we  have  “thriv¬ 
ing”  in  our  crosshairs!  Although  we 
must  remain  realistic  about  the  fu¬ 
ture,  our  outlook  for  this  new  year  is 
one  of  excitement  and  opportunity 
for  professional  development.  In 
the  following  paragraphs,  I  describe 
three  noteworthy  areas  in  which 
MORS  has  adapted  and  continues 
to  provide  you  with  career-enhanc¬ 
ing  development  and  collaboration 
opportunities:  the  82nd  MORS 
Symposium,  our  Special  Meetings, 
and  MORS  Local  Chapters. 


Please  join  us  for  this  year’s  pre¬ 
miere  event  for  operations  research 
professionals,  the  82nd  MORS 
Symposium.  For  the  second  year, 
we  are  holding  the  Symposium 
in  our  nation’s  capital,  a  location 
that  is  considered  local  to  about 
half  of  our  membership.  And  for 
the  second  year,  we  are  offer¬ 
ing  significant  virtual  participation 
opportunities  for  those  unable  to 
travel.  Even  better:  the  Symposium 
still  provides  a  rich  forum  for  OR 
professionals  to  present  their  work, 
gain  feedback  from  peers,  learn 
about  other  projects  in  the  com¬ 
munity,  make  and  grow  lifelong 
acquaintances,  and  enjoy  the 
professional  camaraderie  ever 
present  in  our  Society. 


everal  past  Phalanx  articles 
have  emphasized  the  need 
for  our  Society  to  adapt  to 
the  changing  fiscal  environ¬ 
ment  in  order  to  provide 
you  with  the  professional 
services  you  need.  In  fact,  the  title 
of  my  December  article  was  “Learn, 
Adapt,  Survive,  and  Thrive!”  Now 
that  we  are  a  full  year  into  the  ef¬ 
fects  of  sequestration,  budget 
reductions,  and  travel  restric- 


Symposium  Chair  Rochelle  An¬ 
derson,  Working  Group/Compos¬ 
ite  Group  Chair  Sheilah  Simberg, 
Special  Sessions  Chair  Hunter 
Marks,  and  83rd  Symposium  Chair 
Ronda  Syring  are  leading  a  cast  of 
“Energizer  Bunnies”  that  have  put 
together  an  impressive  schedule  of 
educational  and  professional  de¬ 
velopment  events.  That  schedule  in¬ 
cludes  the  full  set  of  virtual  sessions 
conducted  over  Defense  Connect 
Online  (DCO)  June  4-6,  and  the 
main  event,  live  and  in-person,  from 
June  16-19  in  Alexandria,  Virginia. 
Both  live  and  virtual  venues  offer 
unclassified  and  classified  sessions, 
and  allow  participation  by  our  Five 
Eyes  allies.  Be  sure  to  check  out  the 
MORS  website  for  more  details. 

Our  opening  plenary  session  will  be 
a  special  event,  with  Mrs.  Katrina  G. 
McFarland,  Assistant  Secretary  of 
Defense  (Acquisition)  delivering  the 
keynote  address  (see  Mrs.  McFar¬ 
land’s  bio  on  page  26).  Immedi¬ 
ately  following  the  keynote,  we  will 
engage  our  government  sponsors  in 
a  panel  discussion,  with  ample  time 
for  questions  taken  from  the  audi¬ 
ence.  Throughout  the  Symposium, 
professional  development  op¬ 
portunities  abound  with  numerous 
parallel  tracks,  special  sessions, 
and  many  occasions  for  networking. 
Abstract  submission  and  registra¬ 
tion  for  the  Symposium  are  well 
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underway  at  the  MORS  website. 

Also  be  sure  to  see  Rochelle’s  article 
on  page  25. 

This  past  November,  we  conducted  a 
100  percent  virtual  training  workshop 
on  test  and  analysis  techniques. 
Although  we  have  conducted  virtual 
events  in  the  past,  this  was  the  first 
to  do  so  along  a  somewhat  tradi¬ 
tional  MORS  special  meeting  model, 
with  seven  parallel  tracks  running 
continuously  over  two  days.  Con¬ 
gratulations  and  many  thanks  to 
meeting  co-chairs  Don  Timian  and 
Greg  Hutto,  and  their  entire  team 
of  instructors  and  support  staff,  for 
making  this  event  a  great  success  by 
any  measure! 

This  month,  we  are  seeing  an¬ 
other  first  for  MORS:  a  workshop 
that  grew  out  of  a  Community  of 
Practice  (CoP)  that  grew  out  of  a 
workshop  that  itself  grew  out  of 
a  previous  special  meeting.  Allow 
me  to  explain.  In  September  201 1 , 
MORS  held  the  workshop,  “Risk, 
Trade  Space  &  Analytics  in  Acqui¬ 
sition”  to  identify  and  share  best 
practices  for  acquisition  analysis. 
One  significant  conclusion  from  that 
workshop  was  that  “affordability 
analysis”  was  poorly  defined  across 
the  community.  To  address  that 
shortcoming,  MORS  held  a  follow- 
on  workshop  entitled,  “Affordability 
Analysis:  How  Do  We  Do  It?”  in 
October  2012.  A  chief  finding  from 
that  workshop  is  that  the  topic  is 
sufficiently  complex  and  important 
to  demand  the  establishment  of  a 
CoP  that  would  meet  regularly  and 
build  up  the  body  of  knowledge 
around  affordability  analysis.  MORS 


established  the  Affordability  Analy¬ 
sis  CoP  in  February  2013  under  the 
leadership  of  Kirk  Michealson,  FS, 
and  it  quickly  became  both  very  ac¬ 
tive  and  very  productive. 

In  fact,  the  Affordability  CoP  has 
been  so  productive  that  not  only  has 
it  met  monthly  during  the  past  year, 
but  it  has  also  produced  a  written 
summary  of  its  work  to  date  in  the 
form  of  an  “Affordability  Research 
Document”  (available  on  the  MORS 
website)  and  has  planned  and  exe¬ 
cuted  a  second  follow-on  workshop. 
This  event,  “Affordability  Analysis: 
Developing  the  Process,”  is  being 
held  March  10-12,  2014  at  the  Lock¬ 
heed  Martin  Global  Vision  Center  in 
Crystal  City,  Arlington,  Virginia.  Also 
of  note  is  the  ongoing  tie  between 
the  Affordability  CoP  and  our  annual 
Symposium’s  Composite  Group 
E  (Acquisition),  Working  Group  26 
(Cost  Analysis),  and  Working  Group 
27  (Decision  Analysis). 

The  next  exciting  innovation  in 
special  meetings  this  year  is  a  risk 
analysis  workshop  jointly  spon¬ 
sored  by  MGRS  and  the  Security 
Analysis  and  Risk  Management 
Association  (SARMA).  Like  MGRS, 
SARMA  is  a  nonprofit  professional 
association.  Unlike  MORS,  SARMA 
intentionally  focuses  on  analyz¬ 
ing  and  managing  security  risks  to 
complex  systems  from  man-made 
threats.  This  partnership  between 
MORS,  with  its  broad  mission  to 
enhance  national  security  analysis 
and  advance  the  OR  profession, 
and  SARMA,  with  its  specialized 
mission  to  provide  risk  analysis  and 
policy  advice  to  decision  makers, 
brings  together  two  strong  and 


complementary  communities  to 
work  on  an  important  problem. 

Set  for  May  5-6  at  the  MITRE  Cam¬ 
pus  in  McLean,  Virginia,  MORS  and 
SARMA  will  conduct  the  “Risk  Anal¬ 
ysis  in  the  Logistics  &  Acquisition 
Supply  Chain”  workshop.  Partici¬ 
pants  in  this  event  will  examine  best 
practices  for  the  significant  analytic 
challenges  in  assessing  risks  within 
the  supply  chain.  These  topics 
are  not  new,  but  their  importance 
continues  to  intensify  as  our  move 
into  the  cyber  and  space  domains 
continues  to  accelerate  at  a  fantastic 
pace.  Because  of  this,  system  and 
subsystem  interdependencies  and 
supply  chain  vulnerabilities  have 
become  harder  to  identify,  quantify, 
and  analyze. 

The  meeting  leadership,  including 
co-chairs  Jim  Muccio  and  Scott 
Berg,  Bulldogs  Jim  Bexfield,  FS, 
and  Paul  Byron  Pattak,  and  Advisor 
Renee  Carlucci,  has  developed  an 
exciting  and  unique  program.  The 
workshop  will  focus  on  supply  chain 
problems  within  three  communities: 
homeland  security,  defense,  and  in¬ 
telligence.  This  will  be  done  in  three 
parallel  tracks,  and  at  three  different 
levels  of  classification:  unclassified, 
secret,  and  top  secret.  Be  sure  to 
visit  the  MORS  website  for  more 
information  about  the  meeting  struc¬ 
ture  and  security  requirements,  and 
to  register  for  the  workshop. 

The  1 1th  Annual  Education  and 
Professional  Development  (EPD) 
Colloquium,  to  be  held  April  28-29, 
2014  at  the  Virginia  Military  Institute 
in  Lexington,  Virginia,  is  perhaps  our 
most  ambitious  EPD  Colloquium 
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ever.  Thanks  to  CDR  Harrison 
Schramm  and  his  energetic  team, 
we  have  an  abundance  of  events, 
inciuding  the  Rosenthai  Competi¬ 
tion,  in  which  coiiege  students 
coiiaborate  on  a  quick-turn  anaiysis 
and  have  their  presentations  judged 
by  a  panei  of  nationai  security 
anaiysis  experts;  a  student  poster 
competition,  in  which  students  are 
avaiiabie  to  discuss  their  projects; 
a  “speed  mentoring  session,”  in 
which  young  anaiysts  interact  with 
senior  anaiysts  in  a  quick  round- 
robin  format;  a  young  anaiyst  panei 
to  heip  address  deveiopment  needs 
of  young  professionais;  a  depioyed 
anaiyst  panei  to  iearn  from  recent 
anaiytic  experiences  in  Afghanistan 
and  potentiaiiy  other  depioyed  ioca- 
tions;  and  a  job  fair  to  engage  ana¬ 
iysts  and  prospective  empioyers. 
See  the  MORS  website  for  more 
information  and  to  register.  Regis¬ 
tration  is  free,  but  space  is  iimited! 

MORS  Local  Chapters 

Locai  chapters,  inciuding  student 
chapters,  have  been  around  in  con¬ 
cept  for  at  ieast  a  decade.  But  the 
actuai  estabiishment  and  activity  of 
such  organizations  has  ebbed  and 
fiowed  with  the  needs  and  ambi¬ 
tions  of  iocai  communities.  Ongoing 
travei  restrictions  have  caused  a  re¬ 
newed  interest  in  such  iocai  MORS 
chapters  for  obvious  reasons. 

The  MORS  Rocky  Mountain  Chap¬ 
ter  (RMC)  was  buiit  over  the  past 
two  years  by  a  host  of  enthusiastic 
anaiysts,  inciuding  Lee  Lehmkuhi, 
Maj  Travis  Herbranson,  Maj  Ryan 
McGuire,  2nd  Lt  Romeo  Le,  and  Ka- 
thie  Reece.  The  RMC  meets  regu- 
iariy  to  share  ideas  and  enhance 


individuai  knowiedge  of  aii  things 
reiated  to  OR  and  nationai  security. 
The  chapter’s  goai  is  to  provide  a 
iocai  venue  that  aiiows  members  to 
coiiaborate,  recognize  accompiish- 
ments  within  the  MORS  community, 
and  provide  mentorship  for  younger 
anaiysts.  in  addition  to  the  RMC, 
anaiysts  in  Omaha,  Nebraska,  Or- 
iando,  Fiorida,  and  Washington,  DC, 
are  currentiy  taking  the  eariy  steps 
to  estabiish  iocai  chapters. 

As  you  can  easiiy  imagine,  the 
composition  and  activities  of  any 
individuai  chapter  wiii  depend  on  the 
needs,  ambitions,  and  energy  of  the 
iocai  anaiysts.  Of  course,  the  reaiity 
is  that  without  strong  iocai  ieader- 
ship  the  chapters  wiii  not  survive. 
The  good  news  is  that  the  MORS 
community  has  deveioped  a  strong 
cadre  of  potentiai  chapter  ieadership 
over  our  48-year  history  through  the 
conduct  of  our  reguiar  activities. 

Aithough  the  immediate  impetus  for 
a  MORS  iocai  chapter  might  be  to 
offset  the  hardship  travei  restrictions, 
the  potentiai  benefits  are  varied  and 
many.  Locai  chapters  can  recruit 
from  aii  eiements  of  the  traditionai 
MORS  community,  inciuding  miiitary, 
civiiian,  academic,  and  industry. 
Furthermore,  iocai  chapters  can 
often  better  seek  out  nontraditionai 
or  unique  anaiysts  and  projects  that 
might  not  attract  attention  at  the  na¬ 
tionai  ievei,  but  are  nonetheiess  very 
reievant  at  the  iocai  ievei. 

it  is  important  to  understand  that 
iocai  chapters  are  not  a  coiiection  of 
anaiysts  who  work  together  day- 
in  and  day-out  in  the  same  office. 
Aithough  that  kind  of  professionai 


deveiopment  is  important,  suc- 
cessfui  iocai  chapters  incorporate  a 
variety  of  nearby  organizations  into 
the  chapter.  For  exampie,  there  may 
be  major  miiitary  and  contractor 
organizations  in  an  area  (Oriando),  or 
nearby  academic  institutions  (Oma¬ 
ha),  or  a  iittie  bit  of  everything  (Wash¬ 
ington,  DC).  Such  iocai  institutions 
can  provide  chapter  ieaders  with  rich 
professionai  deveiopment  options. 

Locai  chapters  can  provide  profes¬ 
sionai  networking  opportunities  that 
produce  reiativeiy  strong  and  endur¬ 
ing  reiationships  simpiy  because 
of  physicai  proximity.  And  because 
iocai  chapters  maintain  ties  to  the 
iarger  MORS  community,  those  net¬ 
working  opportunities  stiii  have  the 
potentiai  for  giobai  reach.  As  men¬ 
tioned,  chapters  can  provide  a  good 
forum  for  mentoring,  recognizing 
taient,  and  coiiaboration.  One  often 
overiooked  benefit  is  the  abiiity  for 
iocai  chapters  to  provide  ieadership 
opportunities  for  junior  anaiysts 
who  are  often  disproportionaiiy  af¬ 
fected  by  travei  restrictions.  As  one 
of  the  RMC  organizers,  Maj  Travis 
Herbranson,  said,  “Any  way  you 
siice  it,  it’s  just  a  good  idea!” 

Recognizing  Achievement 

i’m  very  happy  to  conciude  this 
articie  by  reporting  that  our  Board 
of  Directors  met  in  eariy  December 
and  eiected  two  new  Feiiows  of  the 
Society  in  recognition  of  their  sig¬ 
nificant  and  iong-term  contributions 
to  MORS.  Piease  join  me  in  con- 
gratuiating  Dr.  Mark  Gaiiagher  and 
Dr.  Niki  Goerger  on  their  seiection  to 
this  iife-time  achievement!  Mark  and 
Niki  wiii  be  formaiiy  instaiied  as  Fei¬ 
iows  at  our  Symposium  in  June. 
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Dr.  Stephen  R.  Riese _ 

President,  Military  Operations  Research  Society 
stephen.riese@jhuapl.edu 


We  also  elected  Dr.  Roger  Burke 
to  the  MORS  Board  of  Directors 
to  fill  the  vacancy  created  when 
Tim  Hope  stepped  down  earlier  in 
the  fall.  We  congratulate  Roger  on 
his  becoming  our  newest  direc¬ 
tor,  and  we  thank  Tim  for  his  years 
of  service  to  MORS,  and  rest  well 
knowing  that  there  are  many  years 
of  service  yet  to  come. 


As  a  final  note,  remember  that 
this  is  your  Society.  Interested  OR 
professionals  like  you  continue  to 
help  our  Society  deal  with  ongoing 
significant  challenges;  for  example, 
virtual  does  not  completely  replace 
in-person,  and  centrally  located 
meetings  are  not  always  ideal.  I 
encourage  you  to  share  your  ideas 
and  energy  with  your  fellow  profes¬ 


sionals,  as  they  do  the  same  for 
you.  Feel  free  to  contact  me,  our 
CEO  Susan  Reardon,  our  Director 
of  Member  Services  Liz  Marriott, 
our  Volunteer  Coordinator  Kirk  Mi- 
chealson,  FS,  or  any  of  the  mem¬ 
bers  of  our  Board  of  Directors,  to 
find  an  eager  ear  for  your  thoughts, 

See  you  in  June!  ® 


lutes  And  Thanks  Our 

Sustaining  Contributors 


***  Three  Star  Sustaining  Contributors 

Dr.  Donna  W.  Blake 

Mr.  Brian  D.  Engler,  FS 

Mr.  Michael  W.  and  Toni  E.  Garrambone 

In  Memory  of  Mr.  Wilson  M.  Garrambone  (USN)  and  Mr.  Vincent  S.  Garrambone  (USAAF) 
In  Honor  of  Dr.  Robert  S.  Sheldon,  FS  and  Mr.  William  H.  Dunn,  FS 

**  Two  Star  Sustaining  Contributors 

Mr.  James  N.  Bexfield,  FS 
Mr.  William  H.  Dunn,  FS 
Dr.  Stephen  R.  Riese 
Dr.  Julie  A.  Seton 
Mr.  E.  B.  Vandiver  III,  FS 

*  One  Star  Sustaining  Contributors 

Dr.  Dick  F.  Deckro,  FS 

Ms.  Tara  A.  Garrambone 

Ms.  Amoretta  M.  Hoeber 

Dr.  John  R.  Hummel 

Mr.  Brian  R.  McEnany,  FS 

Mr.  Herman  H.  "Touggy"  Orgeron 

Mr.  Donald  H.Timian 

Mr.  Richard  I.  Wiles,  FS 


To  become  a  sustaining  contributor  and  make  your  tax  deductible  contribution  to  the  Society  today,  contact 
Susan  Reardon  at  703-933-9075  or  susan@mors.org. 
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_ William  P.  Fox 

President,  Military  Applications  Society 

wpfox@nps.eclu 


The  new  year  is  upon  us  and 
details  on  2014  events  can 
be  found  on  our  INFORMS 
and  MAS  community  web¬ 
sites  (https://www.informs. 
org/Community/MAS),  but 
here  is  a  quick  summary  of  upcom¬ 
ing  conferences: 

•  INFORMS  Annual  Meeting  2014, 
San  Francisco,  Sunday,  Novem¬ 
ber  9-Wednesday,  November 
12,  at  the  Hilton  San  Francisco 
(http://meetings2.informs.org/ 
sanfrancisco201 4) 

•  Twentieth  IFORS  2014,  Barce¬ 
lona,  Spain,  Sunday,  July  13-Fri- 
day,  July  18,  at  the  Barcelona 
International  Convention  Center 
(www.ifors201 4.org) 

Other  ongoing  efforts  include  de¬ 
veloping  support  arrangements  and 
collaborative  partnerships  with  other 
organizations  and  agencies  for  future 
MAS  conferences,  workshops,  and 
events  (such  as  the  Defense  Strate¬ 
gies  Institute,  www.dsigroup.org),  and 
collaborating  with  other  professional 
societies  (such  as  the  Military  Opera¬ 
tions  Society  [MORS,  www.mors. 
org]  and  the  International  Society  of 
Logistics  [SOLE,  www.sole.org] )  and 
institutes  pursuing  scientific  research 
and  applications.  Please  let  us  know 
your  thoughts  and  ideas,  and/or  if 
would  like  to  contribute  to  these  vari¬ 
ous  initiatives. 

Recently,  I  edited  a  special  issue  of 
the  International  Journal  of  Op¬ 
erations  Research  and  Information 
Systems  (IJORIS,  Volume  4,  Number 
3,  July-September  2013).  Contribu¬ 
tions  to  the  special  issue,  “Modeling 
and  Operations  Research  in  Defense 


Analysis,”  were  based  on  two  ses¬ 
sions  on  the  topic  that  the  Military 
Applications  Society  hosted  at  the 
November  201 0  INFORMS  annual 
meeting  in  Austin,  Texas.  Five  of 
our  presenters’  papers  are  included 
in  this  special  issue.  These  articles 
address  operations  research  and 
mathematical  modeling  for  decision¬ 
making  issues  in  the  government  and 
military.  They  introduce  models  and 
analysis  that  deal  with  a  wide  variety 
of  subjects  from  defense  analysis. 

In  addition  to  my  guest  editors’  pref¬ 
ace,  the  following  articles  from  MAS 
members  appear  in  the  special  issue: 

•  “Modeling  and  Methodology  for 
Incorporating  Existing  Technolo¬ 
gies  to  Produce  Higher  Probabili¬ 
ties  to  Detect  Suicide  Bombers,” 
by  myself,  John  Binstock,  and 
Mike  Minutas; 

•  “Docking  Two  Models  of  Insur¬ 
gency  Growth,”  by  Michael  Jaye 
and  Robert  Burks; 

•  “Modeling  the  Complexity  of 
the  Terrorism/Counter-Terrorism 
Struggle:  Mathematics  of  the 
‘Hearts  and  Minds,”’  by  Chris 
Arney,  Zachary  Silvis,  Matthew 
Thielen,  and  Jeff  Yao; 

•  “Performing  Counter-High  En¬ 
ergy  Laser  Evasive  Tactics,”  by 
Donald  P.  Gaver  and  Patricia  A. 
Jacobs;  and 

•  “Behavior  Selection  Using  Utility- 
Based  Reinforcement  Learning 
in  Irregular  Warfare  Simulation 
Models,”  by  Sotiris  Papadopou- 
los,  Francisco  Baez,  Jonathan  Alt, 
and  Christian  Darken. 

We  received  more  submissions  than 
the  journal  had  room  for,  so  IJORIS  is 


printing  the  following  two  additional 
papers  in  their  next  journal  edition: 

•  “Numerical  Solution  for  a  Tran¬ 
sient  Temperature  Distribution  on 
a  Finite  Domain  Due  to  a  Dither¬ 
ing  or  Rotating  Laser  Beam,”  by 
Tsuwei  Tan  and  H.  Zahn;  and 

•  “A  Network  Approach  to  Iden¬ 
tifying  Military  Fleet  Replace¬ 
ment  Strategies,”  by  Patrick 
J.  Driscoll,  Harry  Newton,  and 
Russell  Mosier. 

IJORIS  is  an  excellent  way  for  all  of 
us  in  the  operations  research  com¬ 
munity  to  share  the  knowledge  pre¬ 
sented  at  our  conferences.  I  encour¬ 
age  both  MAS  and  MORS  members 
to  consider  submission  to  it. 

The  continuing  sequestration,  bud¬ 
get,  and  travel  issues  promise  to 
make  2014  an  interesting  year.  We 
will  not  have  a  Spring  2014  MAS 
conference  because  of  these  restric¬ 
tions.  Let’s  hope  for  some  relief  as 
the  year  progresses.  Working  dili¬ 
gently  behind  the  scenes  is  a  superb 
and  responsive  INFORMS  adminis¬ 
trative  staff. 

Most  importantly,  my  sincere  thanks 
to  you  all— our  MAS  membership,  a 
remarkable  group  of  professionals 
supporting  our  Nation  on  this  unique 
cusp  of  history— for  this  opportunity 
and  privilege— truly  an  honor— to 
serve.  I  am  deeply  grateful  for  your 
continued  support  and  confidence 
throughout  the  journey.  Keep  up  the 
great  ideas  and  hard  work. 

See  you  in  San  Francisco.  ® 
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Education  and  Professional 

Development  Colloquium 

Energizing  Our  Rising  Members 


CDR  Harrison  Schramm,  EPD  Chair,  Harrison.schramm@gmail.com;  and  Major  Aaron  Burciaga  (USMCR), 
EPD  Co-Chair/Rosenthal  Competition  Chair,  adburciaga@gmail.com 
EPD  official  email:  epd@mors.org 


his  year’s  Military  Operations 
Research  Society  Education 
and  Professional  Develop¬ 
ment  Colloquium  (MORS/ 
EPD)  will  be  held  April  28-29 
at  the  Virginia  Military  Insti¬ 
tute,  Lexington,  Virginia.  We’re  excited 
to  partner  with  VMI  this  year.  In  addi¬ 
tion  to  Lexington  being  a  rich,  historic, 
and  beautiful  town,  VMI  is  hosting  the 
colloquium  in  Marshall  Hall’s  Center 
for  Leadership  &  Ethics,  a  new  facility 
and  perfect  venue.  This  year  will  be 
the  1 1th  annual  and  best  EPD  yet. 

Our  keynote  speaker  this  year  is  Mr. 
David  Ochmanek,  Deputy  Assistant 
Secretary  of  Defense  for  Force  De¬ 
velopment.  Mr.  Ochmanek  has  had  a 


distinguished  analytic  career  balanced 
between  military  service,  govern¬ 
ment  service,  and  academia.  We  are 
honored  to  have  his  participation. 

His  full  bio  is  available  at  http://www. 

defense.gov/bios/biographydetail. 

aspx?biographyid=235. 

The  EPD  is  for  you  if  you  are  a  stu¬ 
dent  analyst  (undergraduate),  young 
analyst  (early  career),  or  mid-career 
young-at-heart  analyst). 

For  the  young  analyst  (undergraduate): 

*  EPD  is  specifically  targeted  to 
you.  The  EPD  Colloquium  is  struc¬ 
tured  to  energize  and  bring  under¬ 
graduate  students  into  the  fold. 


The  EPD  is  for  you  if 
you  are  a  student  ana- 
iyst  (undergraduate), 
young  anaiyst  (eariy 
career),  or  mid-career 
young-at-heart  analyst. 


USNA  midshipmen  participating  in  the  Educational  and  Professional  Development  Colloquium 
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Share  your  capstone  coursework 
via  presentation  or  poster  session. 

•  Young  Analyst  panel,  chaired  by 
Angela  Severe  of  Lockheed  Mar¬ 
tin,  features  professionals  who  are 
beginning  their  careers  in  opera¬ 
tions  research  reflecting  on  the 
challenges  and  opportunities  they 
face  in  their  daily  work. 

•  OR  Career  Perspectives  panel, 
chaired  by  CDR  Cory  Culver  of 
the  Joint  Staff,  features  those 
further  along  in  their  careers 
discussing  how  their  OR  back¬ 
grounds  have  affected  both  their 
OR  and  non-OR  jobs. 

■  The  Rick  Rosenthal  Competi¬ 
tion,  an  EPD-only  event  chaired 
by  Major  Aaron  Burciaga,  is  a 
“quick  reaction  analysis”  com¬ 
petition.  Students  are  randomly 
assigned  to  teams  and  given 
a  challenging  OR  problem  to 
collaborate  on.  Teams  have  five 
hours  to  apply  OR  methodologies 
to  resolve  the  issues,  demonstrate 
their  skills,  and  develop  a  team 
solution.  On  the  last  day  of  EPD, 
teams  present  the  results  of  their 
efforts  to  a  judging  panel  in  an 
open  forum.  The  winning  team 
receives  a  special  recognition  by 
MORS,  and  each  member  of  the 
winning  team  receives  an  award. 


Registration  is  free,  but 
space  is  iimited.  Visit 
www.mors.org/EPD. 


•  Speed  mentoring,  which  takes 
places  during  luncheons,  intro¬ 
duces  students  and  junior  analysts 
to  seasoned  and  senior  analysts 
for  brief  mentoring  sessions. 

•  Afternoon  sessions  feature  prac¬ 
ticing  analysts  discussing  issues 
and  opportunities  pertaining  to 
their  fields. 

•  A  first-time  job  fair  will  be  held 
the  evening  of  April  28.  This 
event  will  be  targeted  at  young 
analysts  just  starting  their  ca¬ 
reers.  Although  not  immediately 
applicable  to  service  academy 
students,  it  will  be  interesting  and 
beneficial  for  them  to  see  what 

is  out  there  as  they  develop  their 
own  career  roadmaps. 

If  you’re  a  young  analyst  (early  career) 
or  young-at-heart  (mid-career)  ana¬ 
lyst,  you  can 

•  Present  your  analytic  work  or 
resume,  and  share  insights  from 


your  career. 

•  Volunteer  to  serve  as  a  panelist 
or  judge.  If  you  are  interested  in 
serving  in  one  of  these  capaci¬ 
ties,  please  let  us  know  at  epd@ 
mors.org. 

•  Dial  in  to  the  “State  of  Navy’s 
Analytic  Community,”  a  presenta¬ 
tion  by  CDR  Cory  Dixon  of  the 
Navy’s  Operations  Analysis  direc¬ 
torate  (N81). 

•  Learn  about  and  provide  feed¬ 
back  on  certification  opportunities. 

•  Attend  the  job  fair.  This  may  be 
particularly  useful  for  those  at  the 
“8-10  year  crossroads.” 

This  year’s  EPD  will  be  an  opportunity 
for  all  members  of  our  community 
to  build  our  future.  By  learning  what 
young  analysts  are  learning  and  think¬ 
ing,  we  will  be  better  able  to  energize 
new  analysts  and  re-energize  ourselves 

Register  soon!  Registration  is  free,  but 
space  is  limited.  Be  sure  to  visit  us  at 
www.mors.org/EPD  for  updates  and 
important  information  about  travel,  ho¬ 
tel  recommendations,  and  registration. 

Please  email  us  at  epd@mors.org  with 
any  questions  about  the  event. 

We  look  forward  to  seeing  you  at  VMI!  ® 


THE  SECURITY  OF 

PARTNERSHIP 

In  our  increasingly  interconnected  world,  the  concept  of  global  security  has  come  to  encompass  a  broad 
spectrum  of  challenges.  Modern  leaders  look  to  companies  with  strong  experience  in  advanced  technology  and 
wide-ranging  capabilities  to  solve  complex  challenges.  Our  global  team  partners  closely  with  our  customers 
and  approaches  each  mission  and  challenge  as  our  own,  whether  we're  supporting  defense  modernization 
programs,  ensuring  energy  and  economic  security,  protecting  vital  networks  from  cyber  attack  or  launching 
satellites  into  orbit.  We  are  problem  solvers,  applying  rigorous  operations  research  analysis  practices  towards 
all  facets  of  our  business  and  ensuring  we  answer  the  "why"  before  "what"  or  "how."  When  it  comes  to  success 

in  complex  environments,  we  know  partnerships  make  a  world  of  difference. 


www.lockheedmartin.com  LOCKHEED  MARTIN 

We  never  forget  who  we  ’re  working  for®  ^ 


MORS/MAS 

MORS-MAS  Student  Webinar  Series: 

Cugjent  Status 

Future 


Julie  A.  Seton,  Ph.D.,  Indelible  Enterprises,  EEC 

MORS  Vice-President,  Member  and  Society  Services,  julie@indentus.com 


Opportunities  abound  for 

students  and  newly  gradii- 
ated  operations  research 
(OR)  anaiysts  through 
MORS  and  INFORMS- 
MAS.  Society  members 
have  easy  access  to  information,  jobs, 
and  mentoring  reiationships  through 
the  student-centric  webinar  program. 
Any  student,  regardless  of  educational 
program,  school,  or  interests,  may  dial 
into  a  live  webinar  with  topics  focused 
on  OR. 

The  webinar  series  is  an  outreach 
mechanism  for  students  to  engage  in 
OR  professional  societies,  learn  from 
OR  experts,  and  connect  with  other 
members.  It  is  an  achievement  borne 
out  of  a  concerted  effort  by  Dr.  Niki 
Goerger,  COL  Simon  Goerger  (USA 
Ret.),  and  CDR  Walt  DeGrange  (USN) 
to  bring  the  two  organizations  togeth¬ 
er  for  common  benefit.  Dr.  Julie  Seton 
and  CDR  DeGrange  lead  the  webinar 
productions  for  MORS  and  MAS,  re¬ 
spectively.  This  article  summarizes  the 
webinar  series— what  has  been  done, 
what  is  available  on  the  website,  and 
what  is  planned  for  the  coming  year. 


To  date,  seven  webinar  events  and 
one  analyst  panel  discussion  have 
been  conducted  under  this  program, 
with  more  than  1 00  attendees  for  the 
live  events  and  an  unknown  number 
of  additional  offline  attendees  who 
have  viewed  the  recorded  sessions. 
We  have  experimented  with  different 
formats  (Microsoft  PowerPoint,  Adobe 
PDF,  and  live  Internet),  the  number  of 
presenters  per  session  (ranging  from 
a  single  speaker  to  four  speakers  in 
a  session),  different  webinar  software 
programs  (Go-To-Meeting'*''^  by  Citrix, 
WebEx*^'^  by  Cisco,  and  Google  Hang- 
Out'*'^  Google),  and  length  of  presen¬ 
tation  (ranging  from  5  to  60  minutes). 

Interaction  and  discussion  with  the 
audience  have  occurred  via  tele¬ 
phone,  email,  live  text  chat,  and  live 
Voice  over  Internet  Protocol  (VoIP).  At¬ 
tendees  have  been  able  to  participate 
individually  via  telephone  or  computer, 
or  in  small  groups  gathered  around 
a  single  speakerphone  or  computer. 
Each  webinar  topic  was  selected  by 
the  planning  team  for  its  value  in  the 
following  professional  sectors:  aca¬ 
demia,  defense,  government  (nonmili¬ 


tary),  large  industry,  small  business, 
and  nonprofit.  Table  1  lists  the  session 
titles,  the  dates  they  were  held,  and 
speakers.  Speakers  have  come  from 
a  range  of  professional  expertise  and 
all  have  had  experience  in  more  than 
one  professional  sector,  giving  the 
program  a  breadth  that  is  valuable 
to  student  and  professional  analysts 
alike.  Table  2  (page  12)  lists  speak¬ 
ers  (alphabetically  by  last  name),  their 
presentation  titles,  and  the  profes¬ 
sional  sectors  in  which  they  are  ac¬ 
tive.  Readers  can  use  this  information 
to  identify  mentors  and  advisors,  and 
to  expand  their  professional  networks. 

Recorded  sessions  are  accessible  to 
members  of  MORS  and  MAS  at  the 
Web  addresses  below: 

•  http://www.mors.org/events/stu- 
dentwebinars.aspx 

•  http://www.youtube.com/user/ 
WatchMORS/videos 

•  https://www.informs.org/Com- 
munity/MAS/MORS-MAS-Joint- 
Student-Member-Info 
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Table  1 .  MORS-MAS  webinars  held  in  2012-201 3. 


- 1 

Date 

1 

Webinar  titles  and  speakers 

Speakers 

July  20,  2012 

Introduction  to  MORS-MAS  Student  Membership  Bundling 

Dr.  Julie  Seton  and  CDR  Walt 
DeGrange 

Overview  of  Department  of  Defense  Opportunities  for  OR  Analysts 

Dr.  Jim  Morris 

Overview  of  Academic  Research  Opportunities  for  OR  Analysts 

Dr.  Daniel  Behringer 

September  21, 

2012 

Prize  Winners  In  Operations  Research:  Experiences  and 

Perspectives 

Dr.  Jim  Morris  and  Major  Brady 
Vaira  (USAF) 

November  30, 

2012 

How  Does  Industry  Use  OR  to  Support  the  Military? 

Mr.  Jack  Levis  and  Mr.  Norm 
Reitter 

January  18, 2013 

Strategies  for  Public  Speaking:  Setting  Considerations 

Dr.  Julie  Seton 

March  29,  2013 

A  Medley  of  Military  Problems 

Dr.  Alexandra  Newman 

April  17,  2013 

MORS  EPD  Colloquium  Young  Analyst  Panel 

August  9, 2013 

Mentorship  and  Online  Networking 

Mr.  Dennis  Baer  and  CDR  Walt 
DeGrange 

September  20, 

2013 

Why  and  How  to  Get  Published 

CDR  Harrison  Schramm,  Dr. 
Richard  Deckro,  and  Ms.  Lynda 
Liptak 

Contributing  to  Phalanx 


-  Phalanx  \s  the  magazine  of  MORS  and  MAS,  providing  readers  information  on  the  societies' events, 
professional  information  and  updates,  and  news  that  directly  relates  to  your  career  asjn  operatifift5.anaJ)ist__ 
'  in  the  field  of  national  defense.  We're  looking  for  contributions  to  the  magazine  in  the  following  areas: 


Articles  (up'to  3,000  words)  on  professional  techniques  and  processes  that  have  been 
used  in  practice  "  - 

Commerrtary  on  past  articles  in  the  magazine  or  on  professional  issues  important  to  our  members 


■ 


^  m 


Updates  on  oiif  members' professional  activities  —  what  you're  working  on,  and  your  accomplishments 


Phalanx reaches  the  leaders  of  the  national  security  analytic  community  in  government,  industry,  and 
academia  and  provides  a  unique  forum  for  analysts  to  share  and  learn.  Use  Phalanx  to  share  your  work,  your 
ideas,  and  your  achievements.  Submissions  to  Phalanx  can  be  made  to  phalanx(a)mors.org. 
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Table  2.  Speakers,  titles,  and  professional  sectors. 


Speakers  (listed 
alphabetically)  / 
Professional  sector 


Presentation  titles 
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Mr.  Dennis  Baer  (FS) 
US  Naw 


Mentoring  Is... 


Dr.  Dan  Behringer 
Argonne  National  Laboratory 


Overview  of  Aca(dennic 
Research  Opportunities  for 
OR  Analysts 


Dr.  Dick  Deckro  (FS) 
Air  Force  Institute  of 
TechnolOflif 


Pointers  on  Getting 
Published 


CDR  Walt  DeGrange 
US  Navy  /  Naval  Postgraduate 
School /MAS 


An  Introduction  to 
Operations  Research 
Professional 

Organizations 
Online  Networking 


Mr.  Jack  Levis 
UPS /INFORMS 


INFORMS  and  Analytics 


Ms.  Lynda  Liptak 
Applied  Research  Associates 


Publishing  With  MORS 


Dr.  Jim  Morris 
US  Air  Force 


Operations  Research  in 
the  Federal  Government 
and  Armed  Forces:  OR  in 
the  service  of  our  country 

Prize  Winners  In 
Operations  Research: 
Experiences  and 
Perspectives 


Dr.  Alexandra  Newman 
Colorado  School  of  Mines 


A  Medley  of  Military 
Problems 


Mr.  Norm  Reitter 
Reitter  Consulting 


How  Does  Industry  Use  OR  to 
SuHport  the  Militay? 


CDR  Harrison  Schramm 
US  Navy 


Presentations  and  Publishing 


Dr.  Julie  Seton 
Indelible  Enterprises 


Military  Operations 
Research  Society  Overview 

Public  Presentation 
Strategies  Based  on  Setting 


Major  Brady  Vaira 
US  Air  Force 


Prize  Winners  In  Operations 
Research:  Experiences  and 
Perspectives 
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Plans  for  201 4 

Moving  into  2014,  the  webinar 
program  is  managed  by  the  Educa¬ 
tion  and  Professionai  Deveiopment 
Committee  under  CDR  Harrison  Sch¬ 
ramm.  The  focus  wiii  shift  to  inciude 
junior  anaiysts  as  weii  as  students. 
The  MORS  Junior  Anaiyst  program 
is  now  formaiized  under  the  capabie 
hands  of  Ms.  Angeia  Severe,  with 
guidance  from  MORS  Membership 
Chair,  Mr.  Norm  Reitter.  The  pianning 
committee  wiii  expand  to  inciude 
at  ieast  one  student  and  one  junior 
anaiyst.  We  are  happy  to  have  any 
Phalanx  reader  contact  any  of  us  to 
voiunteer  as  a  pianner,  a  speaker,  or 
an  idea  generator. 

Potentiai  topics  currentiy  under  con¬ 
sideration  inciude: 


•  internship  opportunities 

•  Getting  invoived  with  MORS 
Communities  of  Practice 

•  Abstract  writing  for  conferences 

•  “Must  read”  iist  for  OR  anaiysts 

•  Preview  of  the  MORS  Education 
and  Professionai  Deveiopment 
Coiioquium 

•  Do’s  and  don’ts  of  poster  presen¬ 
tations 

•  Shifting  miiitary  focus:  OR  tech¬ 
niques  to  address  redepioyment 
of  forces  overseas 

•  Presentation  preparation/rehears- 
ais  for  MORSS 

•  Comparing  prizes  and  awards: 
What  are  they?  How  do  i  get 
nominated? 

Look  for  specific  announcements 
regarding  topics,  speakers,  caii-in 
numbers,  and  URLs  that  wiii  come  out 
about  two  weeks  before  each  session. 


Conclusion 

The  MORS-MAS  student  webinar 
series  has  provided  a  wealth  of  infor¬ 
mation  to  our  combined  Societies.  The 
recorded  versions  of  these  sessions 
are  found  on  the  MORS  and  MAS  web¬ 
sites  and  are  available  to  members. 

You  know  students  and  analysts  who 
might  benefit  from  the  existing  ses¬ 
sions.  Suggest  they  become  members 
so  they  can  benefit  from  the  expertise 
and  knowledge  available  online.  You 
also  know  students  and  analysts  who 
will  benefit  from  listening  to  or  pre¬ 
senting  their  work  on  the  prospective 
topics  in  upcoming  webinars.  Suggest 
they  contact  the  MORS  office  or  Dr. 
Julie  Seton  to  schedule  their  par¬ 
ticipation.  Hope  you  can  join  us  for  a 
session  or  two.  ® 
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EXPLORE 

Data  analysis  in  flow 


Introduced  in  1989  with  scientists  and  engineers  in  mind,  JMP*  software  links  powerful  statistics  to  interactive  graphics. 
It  keeps  data  in  flow,  no  matter  whether  it's  small,  tall  or  wide.  Because  there  is  a  graph  for  every  statistic,  you  can 
pursue  your  analysis  without  restraint.  A  sampling  of  its  capabilities: 


•  Regression,  GLM  and  ANOVA 

•  Generalized  Regression:  Ridge, 
Lasso,  Elastic  Net* 

•  Mixed  Models  and  Repeated 
Measures* 

•  Univariate  and  Bivariate  Analysis 

•  Multivariate  Analysis 


•  Data  Mining  Capabilities:  Cross- 
Validation,  Multi-Layer  Neural 
Networks,  Bootstrap  Forests, 
Gradient-Boosted  Decision  Trees, 
Model  Comparison 

•  Nonlinear  Modeling 


•  SAS,®  R,  MATLAB  and 
Microsoft  Excel  Connections 

•  Time  Series  Analysis 

•  Design  of  Experiments 

•  Consumer  and  Market 
Research  Methods 


•  Categorical  Data  Analysis 

•  Reliability  and  Survival  Analysis 

•  Quality  and  Process  Control 

•  One-Click  Bootstrap* 

•  Data  Visualization, 

Mapping  and  Animated  Graphs 

*JMP  Pro  Only 


Try  JMP  sofWvare  for  yourself  at  jmp.com/trial 


ytnp 

Statistical  Discovery.™  From  SAS.  Available  for  MaC®  and  WindoWS  the  power  to  know® 


SAS  Institute  Inc.  product  or  service  names  are  registered  trademarks  or  trademarks  of  SAS  Institute  Inc.  in  the  USA  and  other  countries.  ®  indicates  USA  registration.  Other  brand  and  product  names  are  trademarks  of  their  respective  companies.  ©  201 3  SAS  Institute  Inc.  All  rights  reserved.  S1 1 1 4990US.1 1 1 3 


he  Second  Annual  MORS 
Industry  Day  will  be  held 
March  13,  2014  at  the 
Crystal  City  Hilton  in 
Alexandria,  Virginia.  Our 
theme  “Building  Govern¬ 
ment  and  Industry  Relations”  will 
bring  together  senior  keynote  speak¬ 
ers  and  panel  members  from  both 
government  and  industry. 

The  morning  session  will  feature  two 
keynote  speakers:  Mr.  Bryan  Harris, 
Director  of  Research  and  Develop¬ 
ment  for  Cyber  Analytics  at  SAS,  and 
a  senior  representative  from  Lock¬ 
heed  Martin.  The  panel,  moderated 
by  MORS  President-Elect,  Dr.  Rafael 
E.  Matos,  WBB,  Inc.,  will  consist 
of  government  and  industry  senior 
members.  Both  sessions  will  address 


the  challenges  and  opportunities  of 
managing  “Big  Data”  in  the  national 
security  environment. 

The  lunch  session  will  feature  a  show¬ 
case  with  exhibitors  that  will  include 
small  businesses  as  well  as  large 
organizations  and  institutions.  This  will 
be  a  great  opportunity  to  network  and 
discuss  the  morning  sessions  with  your 
government  or  industry  counterparts. 
The  afternoon  sessions  will  begin 
with  a  keynote  by  Dr.  Forrest  Crain, 
Director,  Center  for  Army  Analysis, 
followed  by  another  mixture  of  gov¬ 
ernment  and  industry  experts  moder¬ 
ated  by  Dr.  Tom  Allen,  FS,  from  the 
Joint  Staff  (J8).  These  two  events  will 
focus  on  how  the  government  and 
industry  can  work  together  to  more 
effectively  identify  and  use  informa¬ 


tion  and  present  it  to  the  senior  lead¬ 
ership,  in  order  to  produce  better- 
informed  actionable  decisions. 

The  event  will  conclude  with  a  MCRS- 
hosted  (hors  d’oeuvres  and  a  cash 
bar)  networking  event  to  discuss  the 
afternoon  sessions  and  continue  to 
build  the  government  and  industry 
relations  from  the  morning  and  lunch 
sessions.  The  event  registration  is  free 
of  charge  to  participants  with  a  nomi¬ 
nal  charge  for  lunch. 

More  information  on  registration  can 
be  found  at  http://www.mors.org/ 
events/industry-showcase-201 3. 
aspx.  For  more  information  on 
becoming  an  Industry  or  Institutional 
Exhibitor  or  Partner,  please  go  to 
WWW.  mors,  org/i  i  p/.  ® 
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Affordability  Analysis 

From  "How  Do  We  Do  lt?"Workshop 

To 

Developing  the  Process"  CoP  Meeting 

Kirk  Michealson,  FS,  kirk.nnichealson@1 979.usna.conn 


In  the  “Communities  of  Practice: 
Keeping  the  Analytical  Momen¬ 
tum  Going  Year  ‘Round”  article 
from  the  December  2013  issue 
of  Phalanx,  Steve  Nortarnicola, 
MORS  Communities  of  Practice 
(CoP)  committee  chair,  mentioned  that 
MORS  CoPs  were  established  and 
organized  for  various  reasons,  from 
sharing  best  practices  to  working  on 
professional  projects.  The  Affordability 
Analysis  CoP,  which  was  formed  as  a 
result  of  the  MORS  “Affordability  Anal¬ 
ysis:  How  Do  We  Do  It?”  workshop,  is 
tied  to  MORS  Symposium  Composite 
and  Working  Groups.  It  developed 
the  Affordability  Research  Document 
for  government  and  industry,  and  is 
planning  a  follow-on  special  meet¬ 
ing  entitled,  “Affordability  Analysis: 
Developing  the  Process,”  to  continue 
the  recommendations  from  the  initial 
Affordability  Analysis  workshop. 

Formation 

In  discussions  in  the  Development 
Planning  (DP)  working  group  at  the 
September  201 1  MORS  Special 
Meeting  on  “Risk,  Trade  Space, 
and  Analytics  in  Acquisition,”  par¬ 
ticipants  discovered  that  affordability 
analysis  was  ill-defined.  That  is,  for 
those  organizations  that  defined  it, 
there  is  not  a  consistent  definition 
for  affordability  analysis  across  the 
Department  of  Defense.  The  work¬ 
ing  group  recommended  developing 
and  formalizing  affordability  analysis 
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processes,  including  recognizing  the 
difference  between  cost  and  afford¬ 
ability  analyses.  They  also  noted  that 
affordability  analysis  should  include 
mission-based,  portfolio-based,  and 
capability-based  analyses. 

As  a  result  of  these  recommendations, 
MORS  led  a  workshop  on  “Afford¬ 
ability  Analysis:  How  Do  We  Do  It?” 
in  October  2012  to  determine  the 
state  of  the  practice  of  affordability 
analysis.  The  executive  summary  ap¬ 
peared  in  the  December  2012  issue 
of  Phalanx,  and  the  final  report  with 
other  information  from  the  workshop 
is  posted  on  the  MORS  Affordability 
Analysis  workshop  webpage  (http:// 
www.mors.org/events/2012aa.aspx). 
The  workshop  recommended  the  fol¬ 
lowing  next  steps: 

•  Form  a  team  to  continue  working 
on  affordability  analysis. 

•  Complete  the  research  not 
conducted  during  the  3  y2-day 
workshop. 

•  Develop  an  affordability  analysis 
“how-to”  manual/guidebook/pro¬ 
cess. 

•  Pilot  the  manual/guidebook/pro¬ 
cess  on  a  couple  of  projects. 

The  first  next  step  was  completed 
when  the  Affordability  Analysis  (/\A) 
CoP  was  established  in  February 
2013  to  continue  working  on  the  rec¬ 
ommendations  from  the  Affordability 


Analysis  workshop.  Kirk  Michealson, 
FS,  the  Affordability  Analysis  work¬ 
shop  chair,  continued  as  the  AA  CoP 
chair  and  many  of  the  current  CoP 
members  from  across  government,  in¬ 
dustry,  federally  funded  research  and 
development  centers  (FFRDCs),  and 
university-affiliated  research  centers 
(UARCs)  participated  in  the  Affordabil¬ 
ity  Analysis  workshop. 

The  AA  CoP  is  organized  into  various 
groups,  depending  on  function: 

•  Leadership  group:  These  are 
service  representatives  from  the 
Air  Force,  Army,  and  Marine  Corps 
who  have  worked  on  this  topic  in 
the  MORS  Acquisition  and  Afford¬ 
ability  Analysis  Special  Meetings. 
Their  function  is  to  participate  in 
the  AA  CoP  and  confer  with  the 
chair  on  CoP  leadership  decisions. 
The  AA  CoP  is  soliciting  a  Navy 
representative.  If  interested,  please 
contact  Kirk  Michealson,  FS. 

•  Product  development  group: 
This  group  works  on  the  recom¬ 
mendations  from  the  Affordability 
Analysis  workshop  and  develops 
products  for  review. 

•  Review  groups:  These  are  sever¬ 
al  groups  that  review  the  products 
created  and  provide  comments 
and  recommendations  to  the  AA 
CoP  chair: 

o  Core  team:  These  are  CoP 
members  from  across  govern- 
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o 


o 


o 


merit,  industry,  and  UARCs  that 
review  the  products  from  their 
organizations’  perspective  and 
provide  comments. 

COCOMS:  Recently  added, 
several  combatant  commanders 
(COCOMS)  have  provided  rep¬ 
resentatives  to  review  the  CoP 
products  from  their  perspective. 
Industry  marketing  partners: 
During  the  Affordability  Analysis 
workshop,  several  professional 
organizations  joined  MORS  and 
have  stayed  involved  in  the  AA 
CoP  to  review  products  from 
their  organizations’  perspective. 
These  organizations  are  the  Na¬ 
tional  Defense  Industrial  Associ¬ 
ation  Systems  Engineering  (NDIA 
SE)  Division,  the  International 
Council  for  Systems  Engineering 
(INCOSE),  and  the  International 
Cost  and  Estimation  Analysis 
Association  (ICEAA). 
Proponents:  Through  repre¬ 
sentatives  from  the  Office  of 
Secretary  of  Defense  for  Ac¬ 
quisition,  Technology  &  Logis¬ 
tics,  the  Assistant  Secretary  of 
Defense  for  Acquisition  was  the 
Affordability  Analysis  workshop 
proponent.  Based  on  the  results 
and  outbriefs,  both  offices  have 
stayed  involved  with  the  AA 
CoP  as  proponents.  In  addition, 
based  on  discussions  during 
the  J8  Leadership  Outbrief,  the 
Joint  Staff/J8  has  also  joined  as 
a  proponent. 


Through  discussions  with  the  MORS 
Symposium  chairs,  the  Affordability 
Analysis  CoP  chair  has  established 
ties  with  the  Acquisition  composite 
group  (CG  E),  the  Cost  Analysis  work¬ 
ing  group  (WG  25),  and  the  Decision 
Analysis  working  group  (WG  26), 
conducting  joint  sessions  with  them 
at  past  Symposia.  Between  Sympo¬ 
sia,  the  AA  CoP  meets  virtually  the 
first  Wednesday  of  each  month  from 
1 200-1 300  ET  via  the  MORS  telecom 
number.  For  more  information  on  the 
Affordability  Analysis  CoP,  please  visit 
the  MORS  website  at  http://www.mors. 
org/events/affordability-analysis.aspx. 
The  /\A  CoP  is  always  looking  for  new 
members.  If  you  are  interested,  please 
contact  Kirk  Michealson,  FS,  at  kirk. 
michealson@1 979.usna.com. 


Affordability  Research 
Document 

After  the  establishment  of  the 
AA  CoP,  the  Product 
Development  group, 
led  by  Dr.  Lisa 
Oakley- 
Bogdewic 
of  MITRE, 
started 
working 
on  the 

recommended 
next  steps  from 
the  MORS 


Affordability  Analysis  workshop. 
Assisting  Lisa  were  Bob  Koury,  Price 
Systems  (MORS  Secretary);  Steve 
Notarnicola,  Lockheed  Martin  Center 
for  Innovation  (MORS  CoP  Committee 
Chair);  Rick  Null,  Lockheed  Martin 
Aeronautics;  and  David  Schumann, 
SAS.  As  the  AA  CoP  started  working, 
the  AA  CoP  Leadership  and  Product 
Development  teams  realized  that: 


•  No  organization  owns  affordabil¬ 
ity  analysis, 

•  Every  organization  wants  to 
know  what  every  other  organiza¬ 
tion  is  doing  in  the  area  of  afford¬ 
ability  analysis,  and 

•  Every  organization  likes  that 
MORS  is  leading  the  effort  to 
define  affordability 
analysis. 
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For  the  second  next  step,  during  the 
workshop  and  leadership  outbriefs, 
several  recommendations  were  made 
for  additional  research.  The  AA  CoP 
Product  Development  team  created 
the  Affordability  Research  Docu¬ 
ment  (ARD),  which  all  review  groups 
have  reviewed.  The  living  document 
is  posted  on  the  AA  CoP  webpage 
(http://www.mors.org/events/afford- 
ability-analysis.aspx). 

The  ARD  organizes  the  additional 
research  into  the  following  sections: 

•  Introduction:  The  Affordability 
Analysis  CoP  Challenge 

•  Decisions  Supported  by  Afford¬ 
ability  Analysis  at  Different  Levels 

•  Exit  Criteria:  Sufficiency  and  Quality 

•  A  Framework  for  an  Affordability 
Analysis  “How  To”  Manual 

•  Recommended  Next  Steps 

•  Appendices:  Acronyms,  ARD 
References,  Other  Affordability- 
Related  Key  References,  and 
Visualization  Ideas 

Section  4.0  of  the  ARD  discusses  the 
statement,  “every  affordability  analy¬ 
sis,  at  any  organizational  or  scope 
level,  must  have  the  proper  scope,  ad¬ 
dress  requirements  assess  the  related 
baseline,  relay  gaps,  and  duplications, 
assess  courses  of  action  for  change, 
and  evaluate  the  relative  benefits  and 
costs  of  the  alternatives  in  question. 

It  must  coordinate  with  portfolios  in 
the  ‘same  level’  tradespace,  as  well  as 
with  those  in  adjoining  levels.  The  af¬ 
fordability  analysis  must  leverage  the 
institutional  data  and  process  artifacts 
and  be  conducted  in  accordance 
with  the  standard  resource  manage¬ 
ment  flow  of  the  organization.  It  must 
present  results  tailored  to  the  decision 
needs  of  the  end  user.” 

As  the  research  was  organized,  a 
framework  emerged  (see  Figure  1). 

The  ARD  proposes  “a  twelve-step 
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Figure  1 .  Affordability  Analysis  1 2-Step  Framework 


Twelve  Step  Framework  in  4  Phases  with 
Driving  Questions  and  "Vital"  Questions 


niGHS 

WIMUry  Op«r«<iww  ■■■■■rcli 


Requirements  and  needs 

1 .  What  is  the  mission  and  outcome  I  am  trying  to  impact? 

2.  What  are  the  desired  outcomes?  How  will  I  know  they  are 
achieved? 

3.  Which  capability's  resource  decisions  will  affect  this  desired 
outcome? 


Is  it  needed?  Are 
Requirements 
Properly 
assessed? 

S'l'AH'l'  ■ 

Is  productivity 

1  and  innovation 
incentivized? 

Baseline  and  Gap  assessments 

4.  At  what  level  do  I  resource  this  capability  area?  What  areas  of 
performance  are  going  well?  What  is  killing  my  budget? 

5.  What  are  the  most  critical  capability  gaps  to  close  or 
efficiencies  to  realize? 


Trade-Off  Analysis  and  Evaluation  \ 

9.  How  do  these  COA  compare  in  net  value  to  other  elements  in 
the  portfolio? 

10.  Do  decision  makers  at  other  levels  need  to  be  contacted? 

1 1 .  What  is  the  best-value  mix  of  investments  in  this  portfolio? 

1 2.  Are  we  "better  off"  than  our  prior  portfolio?  Within 
affordability  boundaries? 

Are  BBP  2.0  initiatives 
ieveraged? 

Is  it  worth  the  cost? 


I  Alternative  Analyses  and  Valuation  Assessments 

6.  What  are  choice  approaches  or  courses  of  action  (COA) 
needed  to  make  needed  changes?  (Strategic,  modernization, 
process,  etc.) 

7.  What  are  the  alternatives  to  consider  for  each  COA? 

8.  How  well  does  each  perform?  What  is  the  risk?  What  is  the 
30-yrcost  profile? 


Are  competitive 
methods  used? 


framework  that  incorporates  end-to- 
end  resource  management  ideals  and 
provides  a  structure  for  affordability 
questions  and  objectives  to  shape 
the  buy-ability  of  desired  capabilities. 
Within  different  phases  of  the  frame¬ 
work,  key  ‘vital’  questions  must  be 
asked  at  all  times  and  be  evident  in  any 
affordability  product.” 

“The  twelve  steps  outline  a  framework 
that  should  be  conducted  at  an  organi¬ 
zational  and  portfolio  level  appropriate 
for  the  decision  at  hand,  and  right-sized 
for  the  efficiencies  or  mission  benefits 
anticipated.  If  decisions  at  one  organi¬ 
zational  level  impact  those  at  another 
level,  these  must  be  accounted  for 
in  the  assumptions  and  data  used. 
Organizations  that  have  active  portfolio 
management  processes  will  be  able 
to  leverage  their  data  and  products  to 
support  affordability.  All  Affordability 
Analyses  should  answer  or  find  an¬ 
swers  to  these  twelve  general  ques¬ 
tions:  if  one  does  not  need  to  conduct 
assessments  to  answer  a  question, 
then  there  should  be  an  answer  avail¬ 
able— the  question  is  still  relevant.” 


step  completed— i.e.,  the  Affordability 
Research  Document— the  AA  CoP 
started  to  plan  how  to  develop  an  af¬ 
fordability  analysis  “how-to”  manual/ 
guidebook/process.  Based  on  previ¬ 
ous  work,  the  AA  CoP  chair  decided 
to  conduct  a  combined  MORS  Special 
Meeting  with  a  Lean  Six  Sigma  Value 
Stream  Mapping  event  to  develop  the 
initial  process. 

The  AA  CoP  meeting  on  Affordabil¬ 
ity  Analysis:  Developing  the  Process 
is  scheduled  for  March  10-12,  2014 
at  Lockheed  Martin’s  Global  Vision 
Center,  2121  Crystal  Drive,  Arlington, 
VA  22202.  Kirk  Michealson,  FS,  the  AA 
CoP  chair,  and  Dr.  Lisa  Oakly-Bogdeic, 
AA  CoP  Product  Development  lead, 
will  co-chair  the  meeting.  The  overall 
goal  of  the  Affordability  Analysis  CoP 
is  not  to  develop  a  prescriptive,  “one- 
size-fits-all”  “how-to”  document  or  a 
manual  on  doing  optimal  resource  allo¬ 
cations,  but  to  develop  an  affordability 
analysis  process  with  best  practices, 
lessons  learned,  considerations,  etc., 
as  well  as  including  ties  to  the  individu¬ 
al  Service’s  new  affordability  policies. 


"How-To"  Manual  Using  the  Affordability  Analysis 

With  the  second  recommended  next  workshop  final  report,  the  Affordability 
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DEFINITION:  4  Lean  Six  Sigma  Value  Stream  transforms  information  into 
a  finai  product/service  for  deiivery  to  the  customer.  The  vaiue  stream  is 
aii  activities  that  provide  value  or  are  required,  but  doesn’t  include  the 
non-value  added  activities.  The  goal  of  a  Lean  Six  Sigma  Value  Stream 
Mapping  event  is  to  identify  these  value-added  and  required  activities 
and  then  map  these  activities  into  an  organized  process. 


Analysis  CoP  Affordability  Research 
Document,  and  other  background 
material,  this  “Affordability  Analysis: 
Developing  the  Process”  meeting  will 
continue  the  work  of  the  Affordability 
Analysis  workshop  and  CoP  and  be 
organized  into  four  working  groups 
with  a  synthesis  group.  The  four  work¬ 
ing  groups  will  be  aligned  with  the  four 
phases  of  the  1 2-step  framework: 

•  WG  1 :  Requirements  and  Needs 

•  WG  2:  Baseline  and  Gap 
Assessments 

•  WG  3:  Alternative  Analyses  and 
Valuation  Assessments 

•  WG  4:  Trade-Off  Analysis  and 
Evaluation 

This  AA  CoP  meeting  will  be  a  com¬ 
bined  MORS  Special  Meeting  and  a 
Lean  Six  Sigma  (LSS)  event  (see  Table 
1).  With  the  facilitation  leadership  of 
LSS  Black  Belts  and  Green  Belts,  each 
of  the  four  MORS  working  groups 
listed  will  conduct  a  Lean  Six  Sigma 
value  stream  map  (VSM)  for  their 
respective  affordability  analysis  phase 
from  the  ARD’s  1 2-step  framework. 

Unlike  other  MORS  Special  Meetings, 
the  final  products  from  this  MORS  AA 
CoP  meeting  will  be  separate  VSMs 
for  each  of  the  affordability  analysis 
phase  working  groups,  with  associ¬ 
ated  input-output  charts.  The  Af¬ 
fordability  Analysis  CoP  will  use  the 


Get  to  Excellence  (Action)  plan  that 
will  also  be  created  in  the  meeting  to 
transform  the  4  WG  VSMs  into  the  af¬ 
fordability  analysis  “how-to”  manual/ 
guidebook/process. 

To  accomplish  this,  the  “Affordability 
Analysis:  Developing  the  Process” 
meeting  will  be  conducted  as  follows: 

•  Monday  afternoon,  March  10: 
Workshop  kick-off/plenary  session. 
From  1300  to  1700,  a  series  of 
foundation  briefs  will  be  presented. 
These  will  include  the  Affordability 
Analysis  workshop  outbrief,  an 
overview  of  the  ARD,  an  update 
of  the  affordability  analysis  defini¬ 
tions,  and  an  overview  of  the  goals 
of  the  workshop.  Then  the  attend¬ 
ees  will  break  into  their  working 


groups  for  introductions  and  learn 
about  their  assigned  phase. 
Tuesday,  March  1 1 :  The  attendees 
will  meet  in  their  assigned  work¬ 
ing  groups  and  start  developing 
their  phase  VSM,  ending  the  day 
with  an  input-output  chart  for  their 
phase.  The  input-output  charts 
will  be  distributed  to  all  working 
groups  during  the  end-of-day 
leadership/integration  meeting. 
Wednesday,  March  12:  In  the 
morning,  the  working  groups  will 
recommence  reviewing  the  other 
phase  working  group  input-output 
charts,  and  will  then  complete 
their  phase  VSMs  and  Get  to 
Excellence  (Action)  plans.  The 
daily  leadership/integration  meet¬ 
ing  will  be  held  during  a  working 
lunch  to  provide  a  last  situational 


Table  1 .  Comparing  a  MORS  Special  Meeting  with  a 
Lean  Six  Sigma  Event 


1  Item 

1  MORS 

LSS^^Iue  Stream  Map 

Daily  meetings 

Integration  meetings 

Management  meetings 

Description  documents 

Concept  paper/TOR 

Charter 

Discussion  facilitation 

If  needed,  WG  chairs 

Black  belts  and  green  belts 

Meeting  product(s) 

Final  report 

Value  stream  map  and  Get  to 

Excellence  plan 

Overview  briefings 

Tutorials 

Background  plenary 

Synthesis 

Synthesis  group 

Project  lead  post-event 

Topic  discussions 

Working  groups 

Affordability  phase  teams 

Working  group  attendees 

Working  group  members 

Team  members 

Working  group  leadership 

Chair 

Team  leader 
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awareness  across  all  working 
groups.  In  the  afternoon,  each 
phase  working  group  and  the 
synthesis  group  will  provide  their 
respective  outbriefs. 

Summary 

There  are  strong  linkages  across  all 
aspects  of  MORS  for  affordability 
analysis:  It  was  recommended  by  the 
September  201 1  Acquisition  Special 
Meeting,  the  AA  CoP  was  recom¬ 
mended  by  the  MORS  Affordability 
Analysis  workshop,  the  AA  CoP  is  tied 
to  MORS  Symposium  composite  and 
working  groups  (CG  E,  WG  25,  and 
WG  26),  and  the  AA  CoP  is  leading 
the  March  10-12,  2014  “Affordability 
Analysis:  Developing  the  Process” 


MORS  meeting.  From  Steve  Notar- 
nicola’s  (MORS  CoP  committee  chair) 
article  on  CoPs  in  the  December  2013 
Phalanx,  the  Affordability  Analysis  CoP 
is  active  is  all  recommended  areas: 

•  Meet  throughout  the  year  on  a 
specific  topic:  Meets  monthly  for 
affordability  analysis. 

•  Continue  discussions  from  the 
Symposium  or  Special  Meeting: 
Formed  to  work  the  recommended 
next  steps  from  the  October  201 2 
affordability  analysis  workshop;  and 
tied  to  the  Acquisition  composite 
group  (CG  E),  Cost  Analysis  work¬ 
ing  group  (WG  25),  and  Decision 
Analysis  working  group  (WG  26). 


•  Share  best  practices:  Will  be 


incorporated  into  the  affordability 
analysis  “how-to”  manual/guide¬ 
book/process. 

•  Work  on  professional  projects: 
The  ARD  and  the  upcoming 
affordability  analysis  “how-to” 
manual/guidebook/process. 

The  Affordability  Analysis  Commu¬ 
nity  of  Practice  is  an  active  MORS 
CoP  providing  value  to  MORS 
customers.  If  you  are  interested  in 
knowing  more  about  the  AA  CoP, 
please  visit  http://www.mors.org/ 
events/affordability-analysis. aspx. 

If  you  are  interested  in  joining, 
please  contact  the  AA  CoP  Chair, 
Kirk  Michealson  FS  at  kirk.micheal- 
son@1979.usna.com.  ® 
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Test  and  .  i  raining 
Analysis  Techniques  Wofr 


Train in 


Ksnop 


Donald  Timian,  US  Army  Test  and  Evaluation  Command,  donald.h.timian.civ@mail.mil;  and  Greg  Hutto, 
US  Air  Force  Materiel  Command,  46th  Test  Wing,  gregory.hutto@us.af.mil 


n  November  18-19,  2013, 
to  an  audience  of  132 
testers  and  analysts,  MORS 
conducted  its  first  virtual 
Defense  Connect  Online 
(DCO)  Training  Workshop. 
Taught  by  recognized  experts  in  the 
field,  classes  on  relevant  topics  such 
as  design  of  experiments  (DoE), 
Bayesian  analysis  methods,  and  com¬ 
puting  statistical  power,  formed  the 
core  of  the  workshop’s  30  tutorials. 

Two  lunch  panels  were  also  conduct¬ 
ed  using  DCO.  On  Monday,  November 
18,  “Department  of  Defense  Perspec¬ 
tive  on  DoE  and  Analysis  Techniques 
Practice,  Policy,  and  Guidance”  was 
presented,  followed  on  Tuesday,  No¬ 
vember  19,  by  “Service  Perspectives 
on  DoE  and  Analysis  Techniques  Ex¬ 
periences,  Policy,  and  Organization.” 


Dr.  Bram  Lillard,  Acting  Science  Advi¬ 
sor,  Office  of  the  Secretary  of  Defense 
Director  Operational  Test  and  Evalu¬ 
ation  (DOT&E),  and  Dr.  Darryl  Ahner, 
Director,  Deputy  Assistant  Secretary, 
Defense  for  Developmental  Test  and 
Evaluation’s  (DASD(DT&E))  Scientific 
Test  and  Analysis  Techniques  (STAT) 
Center  of  Excellence  (COE),  were 
the  lunch  panelists  on  Monday.  Key 
items  discussed  included  the  need 
for  relevant,  credible  evidence  to 
decision  makers  as  driving  factors 
for  requiring  DoE  at  both  DOT&E  and 
DASD(DT&E);  examples  of  flawed 
DoE  application  in  operational  test¬ 
ing;  and  a  short  description  of  the 
DASD(DT&E),  DOT&E,  and  Service 
Test  and  Evaluation  (T&E)  Executives 
signed  scientific  test  and  analysis 
techniques  (STAT  T&E)  policy  and  its 
associated  implementation  plan. 


Lt  Col  Shane  Dougherty  (Air  Force), 
Mr.  Paul  Johnson  (Marine  Corp), 

Dr.  Bob  McIntyre  (Navy),  and  Mr. 

Don  Timian  (Army)  were  the  Ser¬ 
vice  speakers  on  the  Tuesday.  Key 
topics  discussed  included  the  Air 
Force’s  DoE  training  program;  the 
Marine  Corps  Operational  Test  Activ¬ 
ity’s  use  of  DoE  and  its  experiences 
and  lessons  learned  to  date;  the 
Navy’s  12-step  integrated  evalua¬ 
tion  framework/mission-based  test 
design  process  and  how  DoE  fits 
into  step  9,  “Statistical  Design”;  and 
the  Army’s  efforts  to  use  DoE  by 
program  managers  to  find  cost  sav¬ 
ings  while  improving  the  Army  T&E 
statistical  rigor. 

Schedule 

Figure  1  shows  the  schedule  for  the 
workshop.  Most  of  these  classes  will 


Figure  1.  Workshop  schedule.  Most  of  these  classes  will  be  placed  on  the  MORS 
website  for  members  to  access  at  their  leisure. 


Monday  1 8  Nov  2013 

Tuesday  1 9  Nov  201 3 

Engineer  intro  to  DEO  Green  Belt  1  of  4 

[A] 

Engineer  intro  to  DEO  Green  Belt  1  of  4 

[A] 

Replacing  Binomial  Resp.  with  Censored  Continuous  Response 

[D] 

Huntsville  Cases:  Aircraft  and  Systems 

[A] 

GT 

C 

Reliability  Growth  and  Test  Planning  (Continuous) 

[D] 

Reliability  Growth  and  Test  Planning  (Discrete) 

[D] 

’c 

o 

Questionaireand  Rating  Scales  Design  in  T&E 

[B] 

Questionnaire  and  Rating  Scales  Analysis  in  T&E 

[B] 

Fractional,  Min  Abberation  and  Optimal  Designs 

[D] 

Re-Engineering  Existing  Designs  w/DEO 

[B] 

1 .5  Hour  Leading  Change  for  Senior  Leaders  &  Execs 

[C] 

Pickatenny  Cases:  Ammunition  and  Projectiles 

[A] 

Nonnormal  Response  Analiffis:  Gen  Linear  Model  1/2 

[D] 

Nonnormal  Response  Analysis:  Gen  Linear  Model  2/2 

[D] 

Lunch 

OSD  Panel  (DDT&E  -  DOT&E)  -  Practice,  Policy,  Guidance 

[E] 

Services  T&E  Panel  -  Experience,  Policy,  Organization 

[E] 

Engineer  intro  to  DOE  Green  Belt  2  of  4 

[A] 

Engineer  intro  to  DOE  Green  Belt  4  of  4 

[A] 

Introduction  to  R  and  R  Packages 

[A] 

Bayesian  Analysis  Methods  -  Basic 

[A] 

c 

o 

Designs  With  Order  Restrictions  -  Split  Plots 

[D] 

Design  and  Analysis  of  Computer  Experiments 

[A] 

o 

c 

Empirical  Models,  Observational  Studies,  Quasi  Exp  Designs 

[B] 

Reliability  Data  Analysis  with  Censored  Data 

[D] 

Qj 

Computing  Statistical  Power  -  Normal  and  Bi naff  Responses 

[D] 

Developing  a  DT&E  Evaluation  Framework 

[B] 

Design  for  Software-intensive  Systems  1  of  2 

[B] 

Design  for  Software  -  Intensive  Systems  2  of  2 

[B] 

Tutorial  Lessons  -  Poll®  Guidance, Training  Education 

[E] 

Tutorial  Lessons  -  Best  Practices  and  Organization 

[E] 
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be  placed  on  the  MORS  website  for 
members  to  view  at  their  leisure. 

Takeaways:  Test  and  Analysis 
Techniques 

Although  classical  design  and  analy¬ 
sis  of  experiments  apply  unalloyed  to 
a  great  deal  of  Department  of  De¬ 
fense  (DoD)  test  and  analysis  endeav¬ 
ors,  there  is  a  rich  field  of  research 
and  a  trove  of  advanced  techniques 
available  for  more  sophisticated  treat¬ 
ment  of  unusual  processes  and  data 
sets.  Furthermore,  the  DoD  is  blessed 
with  an  exceptional  array  of  gifted 
instructors  in  these  techniques,  able 
to  impart  new  skills  to  the  test  and 
analysis  workforce. 


To  recap  some  of  the  highlights  of  the 
28  technical  tutorials  and  four  orga¬ 
nizational  and  policy  discussions,  we 
offer  the  following  summaries. 

“Advanced  Techniques 
Available— Inquire  Within” 

•  Replacing  binomial  (yes-no  or 
success-failure)  response  vari¬ 
ables  with  rich,  continuous, 
valued  metrics  can  result  in  both 
tighter  confidence  intervals  and 
smaller  required  sample  sizes. 
Applications  include  munitions 
test  (replacing  hit-miss  with  miss 
distance)  and  sensor  testing 
(replacing  detect-no  detect  with 


detection  range  or  time). 

•  Design  and  analysis  approaches 
for  deterministic  software-inten¬ 
sive  system  tests  can  yield  highly 
efficient  test  run  matrices  when 
testing  software  products  for 
functionality.  Applications  include 
mission  planning  software,  enter¬ 
prise  logistics  Web  applications, 
and  data  link  communications. 

•  Methods  for  reliability  growth  plan¬ 
ning  and  reliability  and  life  analysis 
help  ensure  fielded  systems  can 
be  supported  and  maintained  in 
mission-ready  status  in  field  use, 
saving  our  operating  commands 
massive  amounts  in  operations 
and  maintenance  funds. 
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Methods  for  design  and  analy¬ 
sis  of  surveys,  questionnaires, 
and  other  subjective-response 
problems  address  cases  in  which 
expert  opinion  is  a  vital  part  of 
judging  system  effectiveness. 
Problems  such  as  system  usabili¬ 
ty,  operator  workload,  and  uniform 
or  equipment  fit  and  comfort, 
often  rely  on  survey  methods  for 
measuring  effectiveness. 
Generalized  linear  models  (GLM) 
find  application  in  cases  in  which 
responses  are  binary  counts  of 
rare  events,  comprise  skewed  and 
heavy-tailed  distributions,  and  other 
nonnormal  theory  cases.  Applica¬ 
tions  benefiting  from  GLM  may 


comprise  as  much  as  10-20  per¬ 
cent  of  military  test  and  evaluation. 
Bayesian  methods  allow  testers  to 
incorporate  common-sense  prior 
information  and  straight  probabi¬ 
listic  likelihood  interpretations  of 
resulting  analyses. 

The  world  of  split  plot  design 
and  analysis  is  suited  to  cases  in 
which  certain  test  conditions  are 
inconvenient,  inefficient,  or  impos¬ 
sible  to  randomize,  such  as  day- 
night,  vehicle  altitude  or  depth, 
changes  in  hardware  configura¬ 
tion,  or  weather  changes.  Increas¬ 
ingly,  research  material,  teaching 
methods,  and  software  make 
applying  split  plot  techniques  in 
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reach  of  the  practitioner. 

On  occasion,  one  must  analyze 
existing  data  sets  without  the 
benefits  of  being  in  control  of 
test  conditions.  Approaches  and 
suitable  cautions  and  caveats 
were  illustrated  when  analyzing 
nondesigned  data  sets— empiri¬ 
cal  modeling,  quasi-experimental 
designs,  historical  analyses,  and 
observational  studies. 

Instructors  addressed  (and  sup¬ 
plied  spreadsheets  for  implement¬ 
ing)  how  to  compute  statistical 
power  when  normal  theory  as¬ 
sumptions  are  clearly  untenable- 
binary  responses,  count  or  rating 
scale  responses. 
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“Organizational  Change: 
Making  Science  of  Test  ‘The 
Way  We  Do  It  Around  Here’” 

•  One  of  the  most  difficult  human 
endeavors  is  making  fundamen¬ 
tal  change  to  organizations  and 
behavior.  Years  of  effort,  extraor¬ 
dinary  leadership,  and  thorough 
training,  empowerment,  account¬ 
ability,  and  equipping  are  needed 
to  make  change  endure. 

•  Suitable  training  is  needed  at 
all  levels  of  command:  senior 
executives,  middle  manage¬ 
ment,  team  leads,  operators,  and 
analysts  and  engineers.  Train¬ 
ing  is  available  through  vari¬ 
ous  means:  Air  Force  Institute 

of  Technology’s  T&E  Graduate 
Certificate  Program;  STAT  Center 
of  Excellence  short  courses;  US 
Army  Materiel  Systems  Analysis 
Activity  short  courses  on  reli¬ 
ability  growth  and  analysis;  Eglin 
Air  Force  Base,  Florida,  short 
courses;  and  vetted,  high-quality 
commercial  short  courses.  For 
details  on  available  training, 
contact  Dr.  Darryl  Ahner  (Director 
STAT  COE,  darryl.ahner@us.af. 
mil)  or  Mr.  Greg  Hutto  (gregory. 
hutto@us.af.mil). 

•  A  number  of  policy  initiatives  are 
underway  in  both  DOT&E  and 
DASD(DT&E),  including  Interim 
DoD  Instruction  5000.02,  “Opera¬ 
tions  of  the  Defense  Acquisition 
System,”  dated  November  25, 
2013,  the  Defense  Acquisition 
Guidebook,  and  the  Test  and 
Evaluation  Master  Plan  Guide¬ 
lines.  Policy  focuses  on  the  goal 
of  well-designed  experiments,  not 
in  mandating  methods. 

•  Case  studies  abound— at 


DOT&E’s  extranet  (www.dote.osd. 
mil),  in  National  Defense  Indus¬ 
trial  Association  and  International 
Test  and  Evaluation  Association 
publications,  at  the  US  Air  Force 
DoE  SharePoint  (https://cs3.eis. 
af.mil/sites/00-TE-MC-79/de- 
fault.aspx),  and  at  this  workshop. 
Contact  Mr.  Greg  Hutto  (gregory. 
hutto@us.af.mil)  or  Mr.  Don  Tim- 
ian  (donald.h.timian.civ@mail.mil) 
for  details. 

•  Experience  shows  that  a  few  key 
hires,  well-educated  and  expe¬ 
rienced  in  experimental  design 
and  analysis,  accompanied  by 
growing  practitioners  from  within, 
works  well  in  many  organizations. 

•  Organizing  analysts  into  a  core 
group  and  matrixing  them  to  line 
projects  allows  both  the  dynamic 
flexibility  needed  by  test  and 
analysis  shops,  and  a  critical  mass 
for  rapid  mentoring  and  seasoning. 

Key  Lessons  Learned:  Virtual 
Workshops 

Given  today’s  fiscal  environment, 
“virtual”  training  and  meetings  will  be¬ 
come  more  frequent.  For  this  reason, 
some  of  the  “lessons  learned”  from 
the  online  workshop  are  worth  noting. 

Meeting  Preparation 

1 .  Decide  on  the  scope  of  the  meet¬ 
ing’s  virtual  component  early. 
Virtual  participants  need  to  be 
familiar  with  the  collaborative 
technology  utilized,  in  this  case 
DCO.  Expert  users  are  needed 

to  provide  participants  and/or 
speakers  with  technical  help. 

2.  Notification  of  the  virtual  event 
must  be  out  at  least  30-60  days 
prior  to  the  event. 


3.  Speakers/instructors  in  a  virtual 
environment  must  be  prepared. 
Most  in  this  workshop  seemed 
comfortable  with  talking  to  a  vir¬ 
tual  audience. 

4.  If  recording,  ensure  that  partici¬ 
pants  are  prepared  to  start  and 
stop  the  recording. 

5.  Suggest  that  interested  individu¬ 
als  participate  from  a  quiet  or 
remote  location  away  from  work. 
Several  people  should  reserve 

a  conference  room  and  take  the 
class  together.  This  could  help 
limit  workplace  distractions. 

6.  Monday  is  a  hard  day  on  collab¬ 
orative  services.  If  possible,  start 
on  a  Tuesday  or  later  in  the  week. 

7.  Give  students  an  early  “check¬ 
in  time”  to  eliminate  machine  or 
browser  issues. 

8.  Have  participants  sign  up  for 
classes  they  plan  to  attend  and 
give  the  instructors  a  rough  num¬ 
ber  of  how  many  to  expect.  This 
will  help  instructors  know  when 
they  have  a  quorum  and  can  start. 

9.  Test  all  systems  well  before  start¬ 
ing  the  meeting.  Have  a  phone 
back  up  to  VoIP. 

10.  Use  a  secure  collaborative 
system  if  you  want  to  restrict 
attendance.  DCO  rooms  are  not 
“secure”;  anyone  who  has  the 
URL  can  attend. 

1 1 .  It  is  good  to  see  who  is  talk¬ 
ing,  and  streaming  video  of  the 
speaker  is  valuable. 

12.  Seek  and  encourage  par¬ 

ticipant  feedback  during  the 
presentation,  by,  for  example, 
posting  questions  to  the  chat 
window  and  having  the  present¬ 
er  cover  them  when  possible. 
Virtual  silence  is  more  deafening 
than  real  silence.  ® 
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Update  on  the82nd  MORS 

Guiding  the  Nation  SympOSium 

through  Uncertain  Times 

Rochelle  Anderson,  US  Army  TRAC,  rochelle.a.anderson.civ@mail.mil 


Planning  continues  for  the 

next  MORS  Symposium,  to 
be  held  virtually  June  4-6, 
2014,  and  in-person  at  the 
Hilton  Alexandria  Mark  Cen¬ 
ter  in  Alexandria,  Virginia, 
June  16-19,  2014.  The  theme,  “Guid¬ 
ing  the  Nation  through  Uncertain 
Times,”  reflects  the  role  of  operations 
research  analysis  within  the  defense 
community  as  our  leaders  grapple 
with  difficult  decisions  in  an  austere 
fiscal  environment.  The  symposium 
team  is  hard  at  work  planning  the 
composite  and  working  groups  (CG/ 
WG),  special  sessions,  tutorials, 
demonstrations,  posters,  plenary 
session,  and  other  details  specific  to 
the  82nd  MORSS. 

Mrs.  Katrina  McFarland,  Assistant 
Secretary  of  Defense  (Acquisition), 
will  be  the  plenary  session  keynote 
speaker  for  the  82nd  Symposium. 

Mrs.  McFarland  is  the  principal  advi¬ 
sor  to  the  Secretary  of  Defense  and 
the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics 
on  matters  relating  to  acquisition.  You 
can  learn  more  about  her  by  reading 
her  biography,  also  published  in  this 
edition  of  Phalanx. 

We  have  a  great  program  planned  for 
the  participants  of  this  year’s  sympo¬ 
sium,  beginning  with  analytic  briefings 
from  across  the  Department  of  De¬ 
fense  analysis  community  and  includ¬ 


ing,  for  the  first  time,  presentations  by 
analysts  from  allied  nations  (Australia, 
Canada,  New  Zealand,  and  the  United 
Kingdom).  The  virtual  symposium 
will  include  classified  and  unclassi¬ 
fied  briefings  based  on  the  type  and 
number  of  abstracts  submitted  during 
the  planning  process. 

The  symposium  week  begins  with 
the  Rist  prize  competition,  tutori¬ 
als,  and  continuing  education  unit 
courses  on  Monday,  June  16.  New 
to  the  program  this  year  will  be  one 
to  three  no-cost,  short  pilot  courses 
based  on  topics  chosen  by  the 
MORS  Sponsors  and  the  results  of 
a  survey  conducted  by  the  MORS 
Continuing  Education  Committee. 
These  are  in  addition  to  two  continu¬ 
ing  education  unit  courses:  “Intro¬ 
duction  to  Analysis  for  Practitioners” 
for  new  analysts,  and  “Introduction 
to  Analysis  for  Study  Leaders”  for 
potential  study  leads. 

Tuesday,  June  17,  continues  bright 
and  early  with  the  CGAA/G  chair 
warm-up  session,  followed  by  the 
plenary  session.  The  Sponsor’s  Hot 
Topics  panel  will  immediately  follow 
the  plenary  session.  The  concurrent 
sessions  for  the  CG/WG  begin  after 
lunch  and  continue  through  Thurs¬ 
day  afternoon. 

The  82nd  MORSS  will  also  feature  a 
full  program  of  posters  and  demon¬ 


strations  showcasing  concepts  and 
implementations.  This  provides  a 
great  opportunity  to  share  your  work 
and  collaborate  with  peers.  Individuals 
who  have  developed  applications  and 
tools  are  invited  to  share  them  through 
a  demonstration  to  the  MORS  com¬ 
munity.  Demonstrations  may  include 
simulations,  spreadsheet  applications, 
modeling  environments,  and  data 
collection  and  analysis  techniques. 
Posters  offer  an  opportunity  to  share 
your  analysis  in  addition  to  or  in  place 
of  a  briefing  to  a  CGAA/G  session. 
Contact  the  demos/posters  coordina¬ 
tor,  Mr.  Eric  Hansen  (eric.hansen@sas. 
com),  if  you  have  ideas,  comments, 
or  questions  related  to  setting  up  a 
demonstration  or  poster  session. 

The  symposium  program  also  in¬ 
cludes  numerous  informative  and 
instructive  tutorials  led  by  an  array  of 
outstanding  educators,  experienced 
practitioners,  and  renowned  subject 
matter  experts  that  begin  on  Monday 
morning.  There  will  be  presentations 
on  topics  that  support  new  techniques 
and  concepts  being  applied  to  analyti¬ 
cal  studies.  The  tutorials  are  free  to 
MORS  members  and  are  $75  for  non¬ 
members  who  would  like  to  attend  the 
Monday  tutorial  sessions.  Individuals 
must  purchase  a  symposium  registra¬ 
tion  and  meet  security  requirements  to 
attend  tutorial  sessions.  Presentations 
on  Monday  range  from  1  -hour  over¬ 
views  to  half-day  in-depth  classes. 
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During  the  week,  tutorials  are  1-hour  in 
length  during  the  lunch  breaks.  Please 
submit  your  tutorial  abstract  online 
through  the  82nd  MORSS  website  and 
contact  the  tutorial  coordinator,  Dr.  Joe 
Adams  (jadams@ida.org),  with  your 
ideas,  questions,  or  comments. 

The  Special  Session  lineup  promises 
to  be  very  interesting,  beginning  with 
the  MORS  Sponsors’  Hot  Topics  ses¬ 
sion  immediately  following  the  plenary 
session  on  Tuesday  morning.  Following 
that  and  throughout  the  remainder  of 
the  symposium  will  be  sessions  whose 
topics  include  the  MORS  Rist  and 
Barchi  prize  winners,  the  Strategist’s 
Corner  chaired  by  Dr.  Theodore  Ben¬ 
nett  (FS),  the  Deployed  Analyst  ses¬ 
sion  featuring  analysts  from  across  the 
Department  of  Defense  discussing  their 
recent  deployment  experiences,  and  the 
MORS  Heritage  Session  chaired  by  Ms. 
Deborah  Ray,  which  always  features 
fascinating  presentations  related  to  the 
history  of  operations  research  in  the 
US  military.  There  will  be  presentations 
about  MORS  workshops  completed 
during  the  months  leading  up  to  the 
symposium,  a  Junior-Senior  Analyst 
Session  chaired  by  CDR  Harrison 
Schramm,  and  additional  military  spon¬ 
sor  sessions  on  special  topics  of  their 
choosing.  In  addition,  for  the  first  time, 
cadets  from  the  United  States  Military 
Academy  will  present  the  winner  of  the 
annual  Hollis  Award,  an  award  present¬ 
ed  annually  to  individual  cadets  or  ca¬ 
det  teams  at  the  United  States  Military 
Academy  in  recognition  of  excellence  in 
military  operations  research  or  systems 
analysis. 

I  hope  that  you  will  make  plans  to  join 
us  at  the  82nd  MORSS.  We  will  have  an 
outstanding  program  filled  with  topics 
of  interest  to  everyone! 


Katrina  McFarland  is  the  Assistant 
Secretary  of  Defense  (Acquisition) 
(ASD(A)). 

In  this  role,  she  is  the  principal  ad¬ 
viser  to  the  Secretary  of  Defense  and 
the  Under  Secretary  of  Defense  for 
Acquisition,  Technology  and  Logis¬ 
tics  on  matters  relating  to  acquisition. 


of  MDA  on  all  acquisition,  contract¬ 
ing,  and  small  business  decisions. 
During  her  tenure,  Mrs.  McFarland’s 
advice  led  to  more  than  $37  billion 
of  sole  source  procurement  activ¬ 
ity  being  opened  up  to  competition. 
Additionally,  her  successful  efforts  to 
centralize  the  acquisition  of  knowl¬ 
edge-based  services  enabled  small 
businesses  to  compete  for  almost 
half  of  the  MDA’s  knowledge-based 
service,  while  reducing  related  pro¬ 
curement  costs.  Other  core  respon¬ 
sibilities  included  the  development  of 
process  activities  and  program  policy 
associated  with  the  execution  of  the 
single  integrated  Ballistic  Missile 
Defense  System  (BMDS)  research, 
development,  and  test  program,  and 
establishment  of  the  Baseline  Execu¬ 
tion  Review  to  ensure  an  integrated 
program  execution  of  the  BMDS  oc¬ 
curred  across  the  baselines  of  sched¬ 
ule,  cost,  performance,  contracting, 
test,  and  operational  delivery. 

Mrs.  McFarland  began  her  civil 
service  career  in  1986  as  a  general 
engineer  at  Headquarters 


Previously,  she  served  as  the  Presi¬ 
dent  of  the  Defense  Acquisition 
University  (DAU),  where  she  con¬ 
tinued  to  build  DAU’s  outstanding 
reputation  as  the  DoD’s  primary 
learning  institution  while  oversee¬ 
ing  the  development  and  expansion 
of  the  acquisition  curriculum  and 
supporting  learning  opportunities 
for  more  than  150,000  members  of 
the  Defense  Acquisition  Workforce. 
Under  her  leadership,  DAU  provided 
practitioner  training,  career  man¬ 
agement,  and  services  to  enable 
the  acquisition,  technology,  logis¬ 
tics,  and  requirements  community 
to  make  smart  business  decisions 
and  deliver  timely  and  affordable 
capabilities  to  the  war  fighter.  This 
included  addressing  the  ever  chang¬ 
ing  Defense  Acquisition  climate  as 
required  by  the  Under  Secretary  of 
Defense’s  (Acquisition,  Technology 
and  Logistics)  “Better  Buying  Power” 
initiatives,  and  the  recent  National 
Defense  Authorization  Act  directions 
and  guidance. 

Prior  to  joining  DAU,  Mrs.  McFarland 
was  the  Director  for  Acquisition  for 
the  Missile  Defense  Agency  (MDA)— a 
position  she  held  since  May  2006.  As 
MDA’s  principal  acquisition  executive, 
Mrs.  McFarland  advised  the  Director 
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Marine  Corps  where  she  was  accred¬ 
ited  as  a  Materiais,  Mechanicai,  Civii, 
and  Eiectronics  Engineer.  In  1990, 
she  was  hired  by  the  Department  of 
National  Defense,  Ottawa,  Ontario, 
where  she  executed  Procurement 
Head  of  Electronics  duties.  In  1992, 
Mrs.  McFarland  returned  to  the  Ma¬ 
rine  Corps— this  time.  Marine  Corps 
System  Command— where  she  was 
responsible  for  the  acquisition  of  the 
USMC  Aviation  and  Ground  Com¬ 
mand  and  Control,  radars/sensors,  air 
defense.  Combat  ID  and  Cooperative 
Engagement  Capability  initiatives. 

She  continued  to  serve  the  Corps 
through  February  2005,  when  she 
concluded  her  duties  as  the  Director, 
Battle  Management  and  Air  Defense 
Systems  (BMADS). 

Mrs.  McFarland’s  accolades  and  ac¬ 
complishments  are  far-reaching.  She 
has  received  awards  for  her  efforts 
in  the  joint  arena  of  CEC,  C2,  and 
Theater  Missile  Defense  integration 
and  received  recognition  for  her  work 
from  agencies  including  Government 
Computing  News.  Her  articles  have 
been  published  in  the  Military  Op¬ 
erations  Research  Society,  American 
Society  for  Computer  Simulation,  and 
the  International  Aeronautical  Engi¬ 
neering  Societies  Proceedings.  She 
has  received  the  Secretary  of  Defense 
Medal  for  Meritorious  Service  Award; 
and  the  Department  of  the  Navy,  United 
States  Marine  Corps,  Commendation 
Medal  for  Meritorious  Civilian  Service. 

In  addition,  she  is  DAWIA  Level-lll- 
certified  in  program  management,  has 
a  professional  engineer  license  and  has 
attained  her  Professional  Management 
Project  (PMP)  certification.  ® 
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We  make  choices 

between  alternatives 
every  day,  usually 
without  performing 
even  simple  decision 
analysis.  Instead,  we 
use  heuristics,  time-proven  men¬ 
tal  shortcuts  that  allow  us  to  reach 
good  decisions  quickly.  Sometimes, 
however,  we  apply  those  heuristics 
inappropriately  and  they  become 
biases  and  fallacies.  One  we  all  seem 
particularly  prone  to  is  the  Goldilocks 
fallacy,  the  misapplication  of  the 
Goldilocks  heuristic. 

The  Goldilocks  heuristic  is  a  catchall 
principle  used  in  numerous  and  vari¬ 
ous  situations.  Generally,  we  don’t 
use  it  consciously;  we  simply  apply  it, 
and  with  good  success.  In  the  pres¬ 
ence  of  conflict,  we  seek  compro¬ 
mise.  We  keep  to  the  middle  of  our 
lanes  when  driving,  without  calculat¬ 
ing  that  this  decision  creates  a  safety 
margin,  minimizing  the  consequences 


if  there  is  a  sudden  change  or  dis¬ 
traction.  And  when  we  see  seven 
predictions  for  a  hurricane’s  path,  we 
intuitively  assign  higher  likelihood  to 
the  intermediate  predictions  and  less 
to  the  extremes,  without  consciously 
invoking  the  Law  of  Large  Numbers. 
All  of  these  mental  shortcuts  have 
some  justification. 

But  we  also  apply  this  Goldilocks  heu¬ 
ristic  in  questionable  ways.  This  article 
focuses  on  the  idea  that,  in  decision 
problems  where  there  are  conflicting 
objectives,  choosing  a  middle  path  of¬ 
ten  fails  to  give  you  the  best  decision, 
and  sometimes  leads  to  the  worst. 

Example:  Survivability 
vs.  Effectiveness 

The  U-2  flies  regular  reconnaissance 
missions  near  the  Demilitarized  Zone 
between  North  and  South  Korea. 
Suppose  tensions  escalated  between 
the  nations  and  the  Seventh  Air  Force 
Commander  asked  whether  the  usual 


flight  path  should  be  moved  closer  to 
North  Korea  (to  get  better  intelligence) 
or  farther  away  (to  ensure  that  the 
plane  and  pilot  are  safe  and  available 
to  collect  future  intelligence). 

The  problem  can  be  framed  as  a  deci¬ 
sion  between  three  alternatives  (move 
close  in,  move  far  back,  or  take  an 
intermediate  path),  with  two  objectives 
(intelligence  and  survivability).  One 
could  evaluate  the  objectives  for  the 
three  alternatives  by  creating  mea¬ 
sures  for  each  and  obtaining  values  by 
modeling  and  examining  past  quality  of 
intelligence.  Such  an  approach  would 
yield  a  matrix  similar  to  Figure  1 . 

One  obvious  next  step  is  to  weight 
the  two  objectives  and  combine  them 
linearly.  In  this  example,  weighting  the 
intelligence  measure  twice  as  heavily 
as  the  survivability  measure  results 
in  the  best  objective  being  “close  in,” 
whereas  weighting  the  survivability 
measure  twice  as  heavily  as  the  Intel- 
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Intelligence 

Survival 

Close  In 

0.80 

0.60  1 

Intermediate 

0.68 

0.62  1 

Far  Out 

0.65 

_ 0.80  1 

Figure  1 .  Decision  matrix  for  the  U-2 
problem. 


Wi=2,Ws=^ 

CM 

II 

II 

Close  In 

2.20 

2.00  1 

Intermediate 

1.98 

1.92  1 

1  Far  Out 

2.10 

_ 2.25  1 

Figure  2.  Linear  combination  of 
objective  measures  for  the  U-2  ex¬ 
ample,  for  two  sets  of  weights. 


Objective  1 

Objective  2 

Alternative  X 

0 

1 

Alternative/ 

yi 

y? 

Alternative  z 

1 

0 

Figure  3.  Normalized  matrix  for 
two-objective,  three-alternative 
problems. 


ligence  measure  results  in  the  best 
objective  being  “far  out.” 

It  turns  out  that,  for  this  contrived 
example,  the  Goldilocks  heuristic— 
blindly  picking  the  middle  alterna¬ 
tive-fails  to  pick  the  optimum  in 
each  case.  In  fact,  in  this  example, 
under  any  set  of  weights,  the  Goldi¬ 
locks  heuristic  always  leads  to  the 
worst  alternative. 

Such  analytical  situations  are  far  from 
rare  in  the  Department  of  Defense 
(DoD),  Department  of  Homeland  Se¬ 
curity  (DHS),  and  private  industry.  De¬ 
cision  makers  find  themselves  forced 
to  make  judgments  in  cases  that  pit 
effectiveness  against  security,  safety 
against  efficiency,  risk  against  cost, 
and  many  others.  For  many  situations, 
an  optimum  does  indeed  lie  between 
the  extremes,  but  for  many  the  opti¬ 
mum  lies  at  one  of  the  extremes. 

Quantifying  the  Fallacy 

How  often  does  an  anti-Goldilocks 
situation  occur  in  such  decision  prob¬ 
lems?  Can  we  put  a  value  on  how 
badly  this  heuristic  fails  us?  We  can 
consider  problems  that  have  been 
normalized  to  the  structure  in  Figure 
2,  a  typical  step  taken  before  weight¬ 
ing  objectives,  and  one  that  does  not 
cause  any  loss  of  generality. 

Further,  fix  the  weight  of  objective  1 
at  1 ,  and  let  b  be  the  weight  of  objec¬ 
tive  2,  again  without  loss  of  general¬ 
ity.  The  question  is:  When  (and  how 
often)  is  the  intermediate  alternative 
the  optimum?  If  the  intermediate 
alternative  is  the  optimum,  then  the 


combined  value  of  the  two  objectives 
for  the  intermediate  alternative  ex¬ 
ceeds  that  of  either  of  the  other  two 
alternatives: 

+  byg  >  1  and  y^  +  by^>b 


These  relationships  can  be  seen 
geometrically.  The  two  blue  lines  in 
Figure  4  represent  the  equality  case 
of  the  two  inequalities  in  the  equation, 
which  intersect  the  axes  at  (0, 1)  and 
(1 , 0),  each  with  a  slope  of  -1/b.  The 
shaded  area  represents  the  points 
that  correspond  to  the  intersection  of 
the  two  inequalities. 

In  the  diagram  in  Figure  4,  b  <  1 ,  in 
which  case  the  binding  constraint  isy., 
+  by^  >  1 ,  and  the  area  of  the  shaded 
region  is  1/2b.  In  the  case  of  b  >  1 , 
the  binding  constraint  isy.,  +  by^  >  b, 
and  the  area  of  the  shaded  region  is 
b/2.  Thus,  if  y^  and  y^  are  selected  at 
random,  the  probability  of  the  interme¬ 
diate  alternative  being  the  optimum  is 

P  (alternative  y  is  optimal)  = 


This  presumes  no  ordering  of  y.,  and 
y^.  In  most  situations,  the  ordering  is 
known.  Suppose  thaty^  >y^,  in  which 
case  either  Figure  5a  or  5b  pertains. 

With  a  bit  more  analytical  geometry 
than  the  unconstrained  case,  it  can 
be  shown  for  this  case  that 

P  (alternative  y  is  optimal)  = 


The  key  observations  are  that,  for  this 
very  common  type  of  decision  prob¬ 
lem,  the  probability  of  the  intermedi¬ 
ate  alternative  being  optimal  depends 
on  the  relative  weights  of  the  objec¬ 
tives.  It  is  at  most  50  percent,  and  it 
could  be  far  less. 

From  an  analytic  standpoint,  use  of  a 
Goldilocks  heuristic  in  such  problems 
can  make  sense  only  in  the  absolute 
absence  of  any  information  relevant 
to  the  decision.  But  even  in  the  ab¬ 
sence  of  the  values  of  the  objectives 
for  each  alternative,  here  is  a  better 
heuristic  than  Goldilocks:  If  b  <  1 , 
then  choose  alternative  z;  and  if  b  > 

1 ,  then  choose  alternative  x.  Only  if  b 
is  precisely  1  (or  there  is  insufficient 
information  available  as  to  the  relative 
weighting)  would  it  be  advisable  to 
choose  the  intermediate  alternative. 
Just  how  much  better  this  heuristic 
is  than  Goldilocks  depends  on  the 
range  of  possible  values  b  might  take. 
The  larger  the  range,  the  worse  Goldi¬ 
locks  fares. 

The  Decision  Process 
as  a  Whole 

Operations  research  addresses  only 
a  portion  of  the  decision  process.  If 
the  goal  is  to  ensure  a  good  decision, 
there  must  be  good  data  and  good 
analysis,  but  the  analysis  has  to  be 
understood  and  believed  as  well.  Mis¬ 
steps  in  presentation  of  the  facts  and 
analysis  can  lead  to  triggering  or  fail¬ 
ing  to  overcome  a  decision  maker’s 
biases  and  fallacies. 

Most  operations  analysts  are  adept 
at  the  first  two  stages  of  Figure  6.  But 
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Figure  4.  Graphical  representa¬ 
tion  of  the  intermediate  alterna¬ 
tive.  The  shaded  region  yields 
values  that  make  the  intermedi¬ 
ate  alternative  optimal, 
our  jobs  often  require  us  to  be  deci¬ 
sion  anaiysts,  adept  at  aii  three.  Yet, 
how  many  of  us  have  taken  a  course 
in  decision  psychoiogy  or  understand 
the  range  of  psychoiogicai  biases  and 
fallacies  that  derail  good  decisions? 

To  understand  the  importance  of 
this  third  stage,  imagine  you  are  a 
decision  maker  for  Customs  and 
Border  Protection  (CBP).  Every  port 
of  entry  is  torn  between  two  objec¬ 
tives:  expedite  legitimate  commerce 
and  interdict  illegal  trafficking.  You’ve 
charged  your  analysts  with  coming 
up  with  a  happy  medium,  the  balanc¬ 
ing  point  between  the  two,  efficiency 
versus  security.  Instead  of  obeying 
that  very  clear  instruction,  they  come 
back  with  some  gobbledygook  about 
how  you  should  either  pursue  one  or 
the  other,  that  any  middle  course  just 
makes  things  worse.  (This  actually  ap¬ 
pears  be  true  across  a  range  of  DHS 
efficiency  versus  security  problems; 
see  Vanden  Bosch  [2013].)  Even  if 
your  analysts  convince  you,  you  can’t 
imagine  convincing  your  boss  that 
you  haven’t  gone  off  the  rails. 

Why  Do  We  Rely  So  Heavily 
on  Goldilocks? 

What  is  it  that  makes  Goldilocks  so 
compelling  that,  even  in  the  face  of 


Figure  5.  Graphical  representations  of  the  intermediate  alternative,  if 
we  know  that  sy^.  The  first  diagram  shows  the  case  for  b  <  1,  and  the 
second  shows  6  >  1,  because  the  shaded  regions  (where  the  intermedi¬ 
ate  alternative  is  optimal)  have  different  geometries. 


good  analysis  suggesting  otherwise, 
one  still  tends  toward  selecting  the 
middle  option  as  best?  Here  are 
some  conjectures. 

Analogy 

As  noted  in  the  introduction,  a  Goldi¬ 
locks  heuristic  works  well  in  many 
areas.  As  a  result,  there  is  a  tempta¬ 
tion  to  apply  it  to  decision  analysis 
or  other  inappropriate  areas  without 
much  conscious  thought. 

Political  necessity 

In  some  situations,  there  is  a  need 
to  assuage  various  parties  by  giving 
each  a  part  of  what  they  want.  In  the 
CBP  example,  for  instance,  analy¬ 
sis  probably  won’t  carry  the  day; 
the  decision  will  be  driven  by  other 
considerations. 

Focus  on  the  worst  case 

Generally,  if  we  exclude  worst  cases 
(those  alternatives  for  which  one  or 
more  objectives  have  lowest  values), 
then  intermediate  solutions  result.  This 
certainly  is  true  of  the  class  of  decision 
problems  analyzed  earlier.  In  many  situ¬ 
ations,  eliminating  the  alternatives  that 
yield  worst  cases  is  prudent,  based  on 
careful  analysis  of  the  decision  maker’s 
value  system.  Sometimes  it  is  not.  In 
this  analyst’s  opinion,  a  choice  of  a 
minimax  regret  decision  process  over  a 


linear  combination  process  should  be 
justifiable  for  reasons  other  than  put¬ 
ting  an  analytical  fig  leaf  on  decision 
makers’  prejudices  against  extremes. 

Monday  morning 
quarterbacking 

Consider  the  U-2  example  again,  but 
from  the  perspective  of  the  next  day, 
after  the  mission.  If  the  decision  had 
been  the  distant  alternative,  and  no 
good  intelligence  had  been  collected, 
would  the  Seventh  Air  Force  Com¬ 
mander  be  criticized  for  his  or  her 
choice?  If  the  decision  had  been  the 
close  alternative,  and  the  plane  had 
been  shot  down,  would  the  Com¬ 
mander  still  have  a  job?  And  does  the 
knowledge  of  these  after-the-fact  con¬ 
sequences  affect  the  Commander’s 
decision?  This  is  closely  related  to, 
but  arguably  distinct  from,  a  focus  on 
worst  case.  (Simonson,  1989,  p.  159). 

Structure  of  the  utility 
functions 

Two  aspects  of  typical  utility  functions 
can  make  the  intermediate  alterna¬ 
tive  more  attractive  than  a  linear  utility 
function  would. 

•  Dwindling  returns.  There  is  a 
well-known  tendency  for  individu¬ 
als  to  value  a  unit  increase  less 
at  higher  values,  resulting  in  a 
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Forget  about  the  porridge,  I  made  Goldilocks  stew 


leveling  off  of  the  utility  curve.  The 
same  effect  is  evident  for  losses. 
The  utility  curve  will  therefore  tend 
to  have  a  “knee”  at  whatever  point 
someone  measures  losses  and 
gains  from.  In  decision  psychol¬ 
ogy,  this  reference  point  is  called 
an  anchor.  (In  Thinking  Fast  and 
Siow,  Daniel  Kahneman  provides 
an  entire  chapter  on  anchoring 
and  its  effects  that  is  worth  every 
analyst’s  attention.) 

Valuing  losses  more  than  gains. 
Tversky  and  Kahneman  (1981) 
also  showed  experimentally  that 
losses  are  valued  twice  as  much 
as  gains.  This  decrease  in  the 
slope  of  the  utility  curve  at  the  an¬ 
chor  also  creates  a  knee  (Kahne¬ 
man  201 1 ,  pp.  284-285). 


When  two  conflicting  objectives  are 
linearly  combined,  and  each  has  a 
knee  near  the  same  place,  there  may 
be  a  local  maximum  or  minimum. 
Individuals  tend  to  choose  inter¬ 
mediate  points  for  anchors,  and 
therefore  there  is  the  potential  for 
that  intermediate  alternative  to  be 
elevated  in  value.  Individuals  create 
these  anchors  in  remarkably  arbitrary 
ways  (Tversky  and  Kahneman,  1981 , 
p.  456).  The  resulting  decisions  are 
therefore  capricious  at  best. 

Cultural  signals 

From  Aristotle  on,  we’ve  been  taught 
that  every  situation  has  a  middle 
ground.  If  you  search  the  Internet  for 
“Goldilocks”  or  “happy  medium,”  you 
will  be  overwhelmed  with  the  positive 


bias  toward  intermediate  alternatives. 
Further,  some  of  us  were  taught  explic¬ 
itly  in  various  service  schools  to  present 
a  decision  maker  with  three  options, 
and  to  place  the  proposed  course  of 
action  between  the  others.  (Instruc¬ 
tors  articulated  this  principle  in  classes 
I  attended,  both  in  Air  Force  Officer 
Training  School  in  1984  and  in  Air  Force 
Squadron  Officer  School  in  1990.) 

This  partial  list  suggests  the  remark¬ 
able  range  of  pressures— practical, 
psychological,  and  political— that 
may  drive  a  decision  maker  or  analyst 
toward  the  intermediate  alternative.  In 
the  face  of  such  pressures,  maybe  it  is 
remarkable  that  we  do  pursue  solu¬ 
tions  analytically,  rather  than  just  trust 
that  the  Goldilocks  heuristic  won’t 
mislead  us. 

Conclusion 

It’s  not  news  that  there  are  fallacies 
and  biases  that  affect  both  analysts 
and  decision  makers,  or  that  these 
can  prevent  good  decisions.  Calling 
this  particular  one  the  Goldilocks 
Fallacy  may  be  new,  but  the  fallacy 
itself  was  recognized  in  the  decision 
psychology  literature  of  the  1970s 
and  1980s  (Simonson,  1989,  p.  158). 
Still,  it  does  not  seem  to  have  been 
addressed  by  the  community  of  op¬ 
erations  analysts.  This  is  remarkable, 
considering  how  often  a  Goldilocks 
heuristic  is  used  and  how  often  it  can 
fail  us  in  the  pursuit  of  a  good  deci¬ 
sion.  This  cursory  analysis  addresses 
a  small  but  significant  subset  of 
problems  and  quantifies  just  how 
poor  the  heuristic  is.  It  also  conjec¬ 
tures  as  to  the  likely  reasons  we  so 
often  turn  to  it  for  fast  assessment  of 
a  situation.  ® 
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omparison 


ifteen  methods  spread  across 
six  publications  with  little  guid¬ 
ance  on  how  to  choose  the 
appropriate  one:  that  is  one 
challenge  faced  daily  by  US 
and  allied  staffs  comparing 
military  courses  of  action  (COAs)  dur¬ 
ing  tactical  and  operational  planning. 
The  effects  of  these  inconsistencies  in 
COA  comparison  are  potentially  cata¬ 
strophic  in  terms  of  lives  lost,  equip¬ 
ment  destroyed,  or  national  objectives 
unachieved  should  the  incorrect  COA 
be  selected  based  on  a  flawed  recom¬ 
mendation.  For  this  reason,  planners, 
commanders,  and  other  decision  mak¬ 
ers  should  reconsider  the  necessity  of 
these  divergent  methods  and  consider 
rewriting  planning  doctrine  to  capture 
the  best  practices  in  multiple  attribute 
decision  making  (MADM)  from  inside 
and  outside  existing  doctrine. 


Background 

The  US  military  and  its  allies  conduct 
tactical  and  operational  planning  fol¬ 
lowing  six  similar  yet  distinct  military 
planning  processes  that  are  listed 
with  their  references  in  Table  1 .  One  of 
these  similarities  is  that  each  planning 
process  includes  a  step  that  analyzes 
and  compares  potential  military  COAs. 
This  step  is  usually  called  COA  com¬ 
parison.  In  this  step,  planning  staffs 
evaluate  COAs  as  discrete,  prede¬ 
termined  alternatives  against  one  or 
more  criteria  (i.e.,  attributes,  goals,  or 
governing  factors)  in  a  MADM  process. 
Most  of  these  processes  recommend 
the  format  of  a  decision  matrix  for  their 
evaluations  as  depicted  in  Table  2. 

Organizing  the  Methods 

Currently,  the  doctrine  of  the  US  Army, 
Marine  Corps,  Navy,  Air  Force,  Joint 
Staff,  and  NATO  each  recommend 


different  decision-making  methods 
to  evaluate  the  COA  comparison 
decision  matrix  in  Table  2.  In  fact,  15 
different  methods  are  recommended 
across  five  of  the  six  publications, 
as  shown  in  Table  3.  The  Air  Force 
is  missing  from  the  comparison  in 
Table  3  because  it  leaves  the  method 
completely  up  to  the  planners  with  no 
recommendations. 

These  15  methods  are  grouped  into 
broad  categories  in  the  third  column 
of  Table  3  to  better  communicate 
their  relationships.  The  “descriptive” 
category  consists  of  methods  that 
are  purely  descriptive  and  qualita¬ 
tive.  The  “additive”  category  groups 
quantitative  methods  that  apply  the 
simple  additive  model  of  adding 
criterion  scores  across  COAs  and 
comparing  the  totals.  The  “additive 
weighting”  category  builds  on  the 

additive  method 
by  employ¬ 
ing  weights  in 
each  category,  a 
MADM  method 
commonly 
known  as  simple 
additive  weight¬ 
ing.  The  “plus 
minus  neutral” 
category  groups 
methods  that  use 
combinations  of 
positive,  nega¬ 
tive,  and  neutral 


Table  1.  Military  planning  processes. 


Organization 

Planning  Process  Name"^^^^ 

Doctrinal  Publication 

US  Army 

Military  Decision  Making  Process  (MDMP) 

Army  Tactics,  Techniques,  and 
Procedures  5-0.1 

USMC 

Marine  Corps  Planning  Process  (MCPP) 

Marine  Corps  Warfighting 
Publication  5-1 

US  Navy 

Navy  Planning  Process 
(NPP) 

Navy  Warfare  Publication 

5-01 

USAF 

Joint  Operation  Planning  Process  for  Air 
(JOPPA) 

USAF  Doctrine  Document  3-0 

Joint  Staffs 

Joint  Operation  Planning  Process  (JOPP) 

Joint  Publication  5-0 

NATO 

Operational  Level  of  the  NATO  Crisis 
Response  Planning  Process 

NATO  ACO  COPDV1.0 
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Table  2.  Example  military  decision  matrix. 


Course  of  Action  (COA) 

Criteria  1 

Criteria  2 

Criteria  3 

COA  1 

COA  2 

COA  3 

Table  3.  COA  comparison  method  classification. 


Organization 

COA  comparison 
method  name 

Type  1 

Broad  category 

US  Army 

Advantages/disadvantages 

Qualitative 

Descriptive 

Unweighted  decision 
matrix 

Quantitative 

Additive 

Weighted  decision  matrix 

Quantitative 

Additive  weighting 

USMC 

Narrative  description 

Qualitative 

Descriptive 

USN 

Nonweighted  numerical 

Quantitative 

Additive 

Weighted  numerical 

Quantitative 

Additive  weighting 

Plus/minus/neutral 

Quantitative 

Plus  minus  neutral 

Advantages  and 
disadvantages 

Qualitative 

Descriptive 

Joint  staff 

Weighted  numerical 

Quantitative 

Additive  weighting 

Nonweighted  numerical 

Quantitative 

Additive 

Strengths  and  weaknesses 

Qualitative 

Descriptive 

Advantages  and 
disadvantages 

Qualitative 

Descriptive 

Plus/minus/neutral 

Qualitative 

Plus  minus  neutral 

NATO 

Advantages  and 
disadvantages 

Qualitative 

Descriptive 

Energy  course  of  action 
(COA)  comparison 

Qualitative 

Enemy  COA 

ratings  to  select  a  preferred  COA. 
Finally,  the  “enemy  COA”  category 
highlights  a  unique  NATO  method 
that  focuses  on  the  enemy’s  COAs, 
rather  than  the  friendly  force’s  COAs. 
Each  organization  addresses  these 
categories  differently,  as  outlined  in 
Table  4.  These  inconsistencies  pose 
challenges  for  the  military  planner 
both  within  his  or  her  own  organiza¬ 
tion  and  when  collaborating  with 
other  organizations.  One  challenge 
within  the  staff  lies  in  choosing  the 
correct  method  for  the  data  type  and 
information  available.  A  separate 
challenge  in  collaboration  is  the  need 
for  standardized  communication  of 
the  method  performed. 


Inconsistencies 

There  are  several  layers  of  inconsis¬ 
tencies  in  how  these  methods  are 
recommended  throughout  doctrine. 
First,  there  is  inconsistency  between 
organizations  on  which  methods  are 
recommended  or  allowed,  as  shown 
in  Tables  3  and  4.  Second,  there 
are  inconsistencies  in  what  criteria 
should  be  evaluated.  Third,  there  are 
inconsistencies  in  how  seemingly 
similar  methods  recommended  across 
organizations  are  implemented.  Each 
layer  is  considered  in  more  detail  in 
the  following  paragraphs. 

The  inconsistencies  in  recommended 
methods  between  organizations  may 


surprise  some  given 
the  interdependent 
and  hierarchical  na¬ 
ture  of  these  organi¬ 
zations’  relationships 
to  one  another.  Oth¬ 
ers  may  see  these 
inconsistencies  as  a 
product  of  unique 
organizational 
cultures,  planning 
in  different  battle- 
space  domains, 
planning  at  differ¬ 
ent  levels  of  war,  or 
some  combination 
thereof.  Regardless, 
the  variance  is  quite 
stark  when  compar¬ 
ing  the  two  extreme 
cases  in  terms  of 
flexibility  repre¬ 
sented  by  the  USAF 
and  the  USMC. 
USAF  doctrine  does 
not  recommend  a 
method  for  COA 
comparison,  leaving 
methodology  selec¬ 
tion  completely  up 
to  the  staff  based 
on  staff  expertise 
and  their  planning 
problem’s  unique 
context.  In  contrast,  USMC  doctrine 
recommends  a  qualitative  narrative  de¬ 
scription  method  and  discourages  any 
quantitative  methods.  The  remaining 
organizations  recommend  or  require 
some  methods  while  giving  staffs  the 
flexibility  to  choose  additional  methods 
not  included  in  their  doctrine. 

The  second  layer  of  inconsistency 
between  methods  is  the  amount  of 
flexibility  given  on  criteria.  Each  organi¬ 
zation,  except  for  the  USAF,  leaves  the 
selection  of  evaluation  criteria  com¬ 
pletely  up  to  the  commander  and  staff. 
What  sets  the  USAF’s  guidance  apart 
is  that  it  prescribes  risks  to  forces  and 
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Table  4.  COA  comparison  methods  by  organization. 


Broad  COA  comparison  method 

Organization 

Descriptive 

Additive 

Additive 

weighting 

Plus  minus 

neutral 

Enemy 

COA 

USA 

Required 

Recommended 

Recommended 

Allowed 

Allowed 

USMC 

Required 

Discouraged 

Discouraged 

Not  addressed 

Not 

addressed 

USN 

Recommended 

Recommended 

Recommended 

Recommended 

Not 

addressed 

USAF 

Allowed 

Allowed 

Allowed 

Allowed 

Allowed 

Joint  Staff 

Recommended 

Recommended 

Recommended 

Recommended 

Not 

addressed 

NATO  1 

Required 

Allowed 

Allowed 

Allowed 

Required 

Table  5.  Unweighted  additive  COA  comparison  methods. 


Organization 

COA  comparison 

method 

Ratm^cale 

Directionality 

USA 

Unweighted  decision 
matrix 

Ordinal 

Less  is  better 

USN 

Nonweighted  numerical 

Interval 

More  is  better 

Joint  Staff 

Nonweighted  numerical 

Ordinal 

More  is  better 

risks  to  mission  as  evaiuation  criteria 
that  should  always  be  used.  These 
criteria  should  be  considered  in  any 
planning  process  and  are  likely  to  be 
considered  by  planners  in  the  other 
organizations,  so  this  inconsistency  is 
the  least  troublesome. 

Of  greatest  concern  are  the  ways  that 
methods  across  organizations  with  the 
same  name  or  similar  methodology  in 
Table  3  are  not  performed  in  the  same 
manner.  Differences  in  the  qualitative 
methods  stem  largely  from  how  the 
advantages  and  disadvantages  of  each 
COA  are  categorized,  but  that  is  pri¬ 
marily  a  difference  in  style  rather  than 
substance.  Differences  in  the  quan¬ 
titative  methods  deserve  additional 
consideration,  beginning  with  the  three 
unweighted  additive  methods.  Table 
5  summarizes  these  differences.  Note 
that  no  two  methods  are  the  same. 

Inconsistencies  in  the  unweighted  ad¬ 
ditive  methods  recommended  by  the 
Army,  Navy,  and  Joint  Staff  center  on 


two  factors  represented  in  the  last  two 
columns  of  Table  5.  First,  there  is  incon¬ 
sistency  in  the  scale  used  for  ratings 
of  how  well  each  COA  scores  in  each 
criterion.  Army  and  Joints  Staff  doctrine 
use  ordinal  rankings  for  their  ratings, 
whereas  Navy  doctrine  prescribes 
interval  ratings.  The  ordinal  scale  allows 
a  planner  to  rank  COAs,  but  not  to 
determine  the  magnitude  of  preference, 
whereas  interval  scales  allow  for  both 
ranking  and  determining  the  magnitude 
of  preference.  The  distinction  between 
these  scales  is  important  because  of 
the  different  mathematical  operations 
that  may  be  applied  to  each  scale  type 
(Stevens,  1946).  The  second  incon¬ 
sistency  lies  in  the  directionality  of  the 
scoring.  The  Army  ratings  are  based 
on  a  “less  is  better”  approach,  whereas 
the  Navy  and  Joint  methods  use  “more 
is  better.”  One  can  see  the  potential 
for  confusion  when  moving  from  one 
organization  to  the  other. 

A  similar  inconsistency  occurs  in  the  dif¬ 
ferent  application  of  the  simple  additive 


weighting  method. 
The  rating  scales 
and  directionalities 
remain  different 
despite  the  com¬ 
mon  use  of  interval 
scale  weights. 

Table  6  summarizes 
these  differences. 
Once  again,  no  two 
methods  are  the 
same. 


Way  Ahead 

This  article  high¬ 
lighted  the  incon¬ 
sistencies  in  the 
MADM  methods 
employed  by  US 
and  allied  staffs  in 
the  analysis  and 
comparison  of 
potential  military 
COAs  during 
planning.  These  inconsistencies  have 
the  potential  to  result  in  inferior  COA 
selection,  resulting  in  reduced  mis¬ 
sion  performance.  Some  critics  have 
argued  that  such  inconsistencies 
between  planning  processes  prevent 
Joint  Staff  officers  from  effectively 
working  together  as  envisioned  in  the 
1986  Goldwater-Nichols  Act  (Ander¬ 
son  and  Slate,  2003).  We  intend  to 
carry  out  the  research  outlined  below 
with  these  concerns  in  mind. 

The  first  research  step  is  to  review 
relevant  literature  with  three  goals 
in  mind.  First,  determine  what  char¬ 
acteristics  make  a  MADM  method 
approachable,  useful,  and  meaningful 
to  a  military  staff.  Second,  use  those 
characteristics  to  find  methods  outside 
of  military  doctrine  that  may  apply 
to  the  planning  processes.  Third  and 
finally,  identify  a  classification  system 
for  the  methods.  This  step  leaves  us 
with  a  comprehensive  list  of  methods 
and  a  means  to  classify  them. 

Continued  on  page  42 
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BIG  DATA 

BIG  DATA  ANALYTICS 

J.  Henningsen,  B.  Cavender,  J.  Muccio,  J.  McQuade,T.  Herbranson,  and  C  Moore 


Dr.  Jaqueline  Henningsen,  SES,  FOS,  is  the  Director  for  Studies  and  Anaiyses,  Assessments  and  Lessons 
Learned,  Headquarters  US  Air  Force.  Dr.  Henningsen  commissioned  a  high  impact  team  in  the  faii  of  2013 
to  brainstorm  on  the  iatest  Big  Data  initiatives  and  DoD  improvement  areas,  for  a  presentation  at  the  MORS 
industry  Showcase.  After  deveioping  the  presentation,  a  breakout  team  continued  to  work  on  the  probiem 
and  crafted  this  articie  to  expiore  the  potentiai  of  Big  Data  in  the  MORS  and  DoD  community. 


In  Carl  Von  Clausewitz’s  semi¬ 
nal  writing,  On  VMar,  he  exten¬ 
sively  describes  the  uncer¬ 
tainty  and  ambiguity  faced  by 
participants  in  military  op-  ^ 
erations.  We  often  charac¬ 
terize  this  uncertainty  as  “the 
fog  of  war,”  and  we  can  extend 
this  application  to  the  chaos  and 
uncertainty  decision  makers  face 
each  day.  As  analysts,  part  of  our 
expertise  is  to  make  sense  out  of 
chaos,  and  data  is  a  foundational 
part  of  the  stories  we  untangle. 
Since  the  dawn  of  the  computer, 
academics  have  measured  the 
growth  and  availability  of  data 
(Press,  2013).  In  the  beginning, 
we  stored  kilobytes  of  data  on 
magnetic  disks  and  used  punch 


cards.  Now,  every  year  the 
world  produces  more  data  than 
previous  years  combined,  and 
the  NSA  is  building  a  yottabyte 


Big  Data”  (Office  of  Science  and 
Technology,  2012),  investing  $250 
million  annually  into  research. 


How  we  can  capitalize  on  the  promises  of  Big  Data? 


storage  facility.  Our  continued 
advances  in  technology  and  data 
analysis  have  led  the  comput¬ 
ing  community  to  embrace  and 
popularize  the  phenomenon 
known  as  “Big  Data”  (Bryant  et 
al.  2008).  As  the  world  transi¬ 
tions  to  the  age  of  Big  Data,  the 
Department  of  Defense  (DoD) 
and  other  government  agencies 
have  recognized  this  opportunity 
and  are  placing  a  “big  bet  on 


As  the  DoD  continues  to  examine 
Big  Data  applications,  technol¬ 
ogy,  analytics,  and  information 
management  techniques,  we 
must  also  understand  how  it  will 
change  our  culture  and  how  to 
capitalize  on  the  promises.  Ac¬ 
cording  to  Gartner  “Big  Data  is 
high-volume,  high-velocity  and 
high-variety  information  assets 
that  demand  cost-effective,  in¬ 
novative  forms  of  information  pro- 
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cessing  for  enhanced  insight  and 
decision  making”  (www.gartner. 
com/it-giossary/big-data).  From 
this  definition,  we  get  the  three  Vs 
of  Big  Data:  volume,  velocity,  and 
variety.  Volume  refers  to  the  sheer 
size  of  the  data  set.  Velocity  deals 
with  how  quickly  the  data  can  be 
made  available  for  analysis.  Variety 
references  the  types  of  data  being 
considered.  The  three  Vs  of  Big 
Data  make  it  clear  that  the  “big”  in 
Big  Data  doesn’t  just  refer  to  size, 
but  also  complexity. 

Most  importantly,  the  def¬ 
inition  of  Big  Data  speaks 
to  the  ultimate  goal  of 
data  analysis:  providing 
the  most  refined  informa¬ 
tion  to  facilitate  decision  mak¬ 
ing.  As  we  seek  more  effective 
methods  of  managing  Big  Data, 
we  must  also  consider  the  ways 
we  will  use  it  to  inform  decisions. 
Whether  we  are  considering 


warfighting,  personnel,  the  plan¬ 
ning,  programing,  and  budgeting 
system  (PPBS),  test  and  evalua¬ 
tion,  or  acquisitions.  Big  Data  will 
play  a  role  in  the  process. 

For  instance,  people  are  the 
most  important  resource  for  any 
organization  and  the  DoD  is  no 
different.  Can  DoD  leverage  Big 
Data  to  help  recruit,  train,  and 
retain  the  best  people  possible? 
Industries  across  the  globe  have 
already  begun  to  utilize  Big  Data 
to  study  “people  analytics”  (Wa- 


ber,  2013).  For  example.  Knack,  a 
Silicon  Valley  start-up  firm,  uses 
computer  games  and  constant 
measurement  to  test  emotional 
intelligence,  cognitive  skills, 
working  memory,  and  propen¬ 


sity  for  risk-taking.  Companies 
such  as  Bain  and  Shell  then  use 
this  information  to  determine 
the  best  people  to  hire.  Alterna¬ 
tively,  consider  applications  in  the 
medical  field.  Currently,  doctors 
view  patient  records  as  a  single 
data  point.  Under  preventative 
health  programs,  with  new  data 
techniques,  doctors  could  fore¬ 
cast  patient  health  indicators  and 
determine  appropriate  treatments. 
This  could  lead  to  a  decrease  in 
overall  healthcare  costs. 


In  another  area,  companies  like 
Progressive  Insurance  provide  a 
model  for  understanding  Big  Data 
applications.  They  are  using  Big 
Data  to  meet  shareholder  de¬ 
mands  and  increase  profit  (SAS, 


The  ability  to  use  Big  Data  provides  the  DoD  with  an 

unprecedented  and  extremely  powerful  capability. 

V _ _ _ J 
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2013).  In  the  DoD,  we  could 
use  Big  Data  to  help  leadership 
navigate  the  array  of  factors  that 
influence  budget  creation  and 
force  optimization.  Big  Data  could 
help  analysts  link  concepts  and  il¬ 
luminate  trade  spaces.  Over  time, 
we  could  leverage  simulations 
to  analyze  program  effective¬ 
ness  to  the  warfighter,  determine 
where  to  continue  investments, 
and  discover  future  requirements. 
From  early  warning  to  stabiliza¬ 
tion,  there  are  countless  op¬ 
portunities  for  the  exploitation  of 
Big  Data.  However,  just  because 
industry  has  adopted  Big  Data  in 
their  day-to-day  operations  does 
not  mean  that  translating  these 
applications  into  military  concepts 
will  be  easy. 

To  create  a  Big  Data  culture  in 
the  DoD,  we  need  to  shift  not 
only  the  way  we  process  data. 


but  also  how  we  think  about 
it.  Decision  makers  and  com¬ 
manders  will  need  to  under¬ 
stand  the  power  of  Big  Data 
and  its  inherent  usefulness  on 
the  battlefield.  In  addition,  we 
must  consider  potential  detrac¬ 
tors,  such  as  how  our  adversar¬ 
ies  could  capitalize  on  Big  Data, 
or  the  security  implications  of 
aggregating  massive  amounts 
of  unclassified  data.  Finally,  the 
benefits  of  Big  Data  will  take 
time  to  mature,  requiring  ad¬ 
ditional  resources  to  implement, 
with  no  tangible  near-term  ben¬ 
efits.  A  continual,  compelling 
story  is  necessary  to  maintain 
the  culture  shift  and  ensure 
strong  leadership  support  in  or¬ 
der  to  maintain  the  culture  shift. 

Big  Data  is  ripe  with  opportunity, 
and  as  analysts  we  need  the  vi¬ 
sion  to  explore  the  possibilities 


and  problems  associated  with 
it.  In  July  2013,  the  White  House 
Office  of  Science  and  Technol¬ 
ogy  directed  federal  agencies  to 
“give  priority  to  investments  that 
address  the  challenges  of,  and 
tap  the  opportunities  afforded  by 
the  Big  Data  revolution”  (Locker, 
2014).  This  mandate  opens 
the  door  to  integration  across 
stovepipes  and  improved  effi¬ 
ciency  and  effectiveness  across 
the  department.  However,  initial 
benefits  are  likely  to  be  intangible 
and  require  strong  commitment 
to  ensure  return  on  investments. 
This  should  not  dissuade  us 
from  embracing  Big  Data  be¬ 
cause  it  provides  the  DoD  with 
an  unprecedented  and  extremely 
powerful  capability. 

If  you  are  interesting  in  sharing  in¬ 
formation  about  projects,  applica¬ 
tions,  and  research  into  Big  Data, 
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please  join  the  DTIC  R&E  Gateway, 
Big  Data  Group.  ® 
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Inconsistencies  Comparison  Continued  from  page  37 


The  second  step  of  the  proposed 
research  will  classify  the  methods 
based  on  the  characteristics  of  the 
problems  that  they  apply  to.  MADM 
methods  must  be  matched  to  prob¬ 
lems  with  the  appropriate  data  types. 
This  leads  to  step  three,  which  will 
screen  the  classified  methods  based 
on  mathematical  legitimacy  and 
mathematical  approachability  for  a 
military  staff.  The  approachability 
aspect  is  important  because  many 
MADM  methods  use  exquisite  calcu¬ 
lations  requiring  significant  computing 
power.  Such  complex  methods  are 
likely  inappropriate  for  staffs  operat¬ 
ing  in  austere  environments  and  rely¬ 
ing  on  personnel  with  varying  degrees 
of  mathematical  fluency.  This  screen¬ 
ing  should  result  in  a  smaller  set  of 
recommended  MADM  methods  for 
military  planning  staffs. 

The  final  step  of  this  proposed  work 
creates  a  visual  decision  support  tool 
for  the  staff.  The  tool  would  allow  a  staff 
to  quickly  choose  from  a  list  of  valid  and 
approachable  MADM  methods  based 
on  their  problem’s  unique  character¬ 
istics,  such  as  availability  of  informa¬ 
tion  and  data  types.  Similar  work  has 
already  been  done  for  MADM  methods 
in  general  (Hwang  and  Yoon,  1981). 


In  conclusion,  although  the  inconsis¬ 
tencies  in  MADM  methods  used  in 
military  planning  may  lead  to  plan¬ 
ning  missteps  now,  they  also  offer  the 
opportunity  to  reassess  their  validity 
and  create  a  tool  for  all  staffs  to  better 
analyze  and  compare  potential  military 
COAs.  In  the  meantime,  staff  mem¬ 
bers  with  a  high  degree  of  mathemati¬ 
cal  fluency  should  guide  staffs  to  the 
most  appropriate  methods  for  their 
particular  planning  problem’s  charac¬ 
teristics  and  context.  This  reinforce  s 
the  need  for  analytical  experts  in  the 
headquarters  of  tactical  and  opera¬ 
tional  units,  and  the  even  greater  need 
for  that  expertise  in  organizations  that 
write  and  review  planning  doctrine.  ® 
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The  31st  International  Sym¬ 
posium  on  Military  Opera¬ 
tional  Research  (ISMOR)  will 
be  held  at  the  Royal  Hol¬ 
loway  campus,  University 
of  London,  England,  July 
29  through  August  1 , 2014.  A  short 
ride  from  Heathrow  Airport,  and  at 
the  successful  site  of  30  ISMOR, 

31  ISMOR  will  feature  an  important 
innovation.  The  Cornwallis  Group 
will  be  present  with  a  different  type 
of  program:  fewer  papers  but  an 
extended  opportunity  for  presenta¬ 
tions  and  interaction  with  the  attend¬ 
ees.  By  agreement  of  Peter  Starkey, 
permanent  chair  of  the  ISMOR  series, 
and  Tony  Hopkin,  the  new  Cornwal¬ 
lis  chair  (replacing  the  senior  founder 
and  long-standing  chair.  Professor 
Dave  Davis),  31  ISMOR  will  introduce 
this  first  great  combination  of  two  of 
the  foremost  institutions  covering  the 
field  of  operational  analysis  applied 
to  national  and  international  secu¬ 
rity  issues  and  the  search  for  peace 
among  nations. 


Present  plans  include  the  Cornwallis 
session  for  the  Wednesday  segment 
of  the  program,  which  will  begin,  as 
usual,  on  Tuesday,  July  29,  2014.  Be¬ 
cause  Cornwallis  takes  fewer  papers 
on  a  single  topic  and  allows  time  for 
in-depth  discussion,  this  represents 
an  attractive  complement  to  the  other 
ISMOR  sessions.  The  Cornwallis  ses¬ 
sion  will  run  as  a  parallel  stream  to 
workshops  and  tutorials.  Wednesday 
will  be  a  workshops  and  tutorials  day, 
with  the  other  days  having  sessions  of 
papers  and  discussions.  Such  ses¬ 
sions  will  be  seeded  with  invited  pa¬ 
pers  to  encourage  discussion.  There 
is  a  possibility  for  a  Friday  (August  1) 
afternoon  session. 


All  the  ingredients  of  the  past  two 
ISMOR  conferences  were  agreed 
upon  as  being  worth  continuing:  short 
papers  on  specified  predetermined 
themes  and  papers  that  do  not  fit 
themes,  one  or  more  keynotes,  invited 
workshops  and  tutorials,  and  a  poster 
session.  For  the  workshop  day,  a 


younger  analysts  and  possibly  stu¬ 
dents  give  short  presentations  on  their 
work  as  a  graduate  briefing  or  peer 
review  session.  Universities  may  be 
invited  to  participate  in  this  session. 


In  addition  to  the  normal  full  residen¬ 
tial  attendance  for  the  week,  flex¬ 
ible  attendance  will  be  offered  as  an 
option  for  31  ISMOR  to  encourage 
participation,  especially  as  many  can¬ 
not  find  time  in  their  schedules  for  the 
whole  event.  To  maintain  the  distinc¬ 
tive  benefits  of  ISMOR  as  an  informal, 
residential  event,  allowing  plenty  of 
time  for  interactions  and  personal 
discussions,  an  appropriate  pricing 
structure  will  encourage  people  to  at¬ 
tend  the  entire  week. 


The  Cornwallis  Group  is  a  spin-off 
from  MORS  and  ISMOR.  It  was 
founded  1 9  years  ago,  at  the  end  of 
the  Cold  War.  The  founders  observed 
that,  with  the  collapse  of  the  Soviet 

Continued  on  page  52 


43 


MORS  Heritage  Pa5^93_i  995 

As  we  continue  the  countdown  to  the  50th  Anniversary  of  MORS,  we  would  like  to  revisit  our  proud  history  and  highlight 
the  past  leaders  of  the  Society  and  key  accomplishments  over  those  years.  Each  edition  of  Phalanx  will  provide  insight 
into  several  years  of  history.  Enjoy  reading  about  these  individuals  and  what  they  have  accomplished.  More  information 
on  the  Past  Presidents,  including  their  Oral  Histories,  can  be  found  on  the  MORS  website. 
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and  James  Sikora,  FS,  formed  a  Mod¬ 
els  and  Simulation  Validation  MORS 
Senior  Advisory  Group. 

1993- 1994:  Jim  Richmann  replaced 
Dee  Ritchie  as  Phalanx  editor. 

1994:  The  following  individual  was 
inducted  as  Fellow  of  the  Society:  Dr. 
Seth  Bonder,  FS. 

1994:  Series  Editor  LTC  Mark  Youn- 
gren  announces  that  the  initial  volume 
of  the  Military  OR  Analyst’s  Handbook 
is  available  for  purchase. 

1994- 1995:  The  United  States  Ma¬ 
rine  Corps  became  the  sixth  MORS 
Sponsor. 


Dr.  Peter  Purdue,  Editor,  Military 
Operations  Research 


1994-1995:  First  year  of  publication  of 
the  MOR  Journal  completed. 

1994-1995:  Dr.  Julian  Palmore  served 
as  Phalanx  guest  editor  for  the  Special 
ADS/DIS  Edition  and,  in  September 
1995,  replaced  Jim  Richmond  as  edi¬ 
tor. 

1995:  The  following  individual  was 
inducted  as  Fellow  of  the  Society: 
Walter  W.  Hollis,  FS. 

1996:  The  following  individuals  were 
inducted  as  Fellows  of  the  Society: 

Dr.  William  G.  Lese,  Jr.,  FS;  James  J. 
Sikora,  FS;  and  E.  B.  Vandiver,  III,  FS. 


MORS  Presidents 


28th  MORS  President:  Gregory  S.  Parnell  (1993-1994) 

Greg  Parnell  served  as  MORS  President  in  1993-1994,  was  elected  a  MORS  Fellow  in  1997, 
was  the  Clayton  Thomas  Laureate  in  2002,  and  received  the  Wanner  Award  in  2013.  He  was 
the  second  editor  of  the  Military  Operations  Research  journal  (1996-2001). 

For  his  undergraduate  studies.  Dr.  Parnell  attended  the  State  University  of  New  York  at  Buf¬ 
falo,  where  he  majored  in  aerospace  engineering.  He  earned  his  master’s  degree  in  industrial 
and  systems  engineering  from  the  University  of  Florida,  and  his  PhD  in  engineering-economic 
systems  (now  called  management  science  and  engineering)  from  Stanford  University. 


Dr.  Parnell  is  a  retired  Colonel  in  the  US  Air  Force  with  25  years  of  service.  He  was  a  professor  of  systems  engineer¬ 
ing  from  the  United  States  Military  Academy  at  West  Point  (1999-2013).  He  was  distinguished  visiting  professor  at  the 
United  States  Air  Force  Academy  (2011-2012)  and  taught  at  the  Virginia  Commonwealth  University  and  the  Air  Force 
Institute  of  Technology. 
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Dr.  Parnell  has  also  served  as  President  of  the  Decision  Analysis  Society  of  the  Institute  for  Operations  Research  and 
Management  Science  (INFORMS).  He  has  also  served  on  three  National  Research  Council  committees. 

He  is  a  Fellow  of  INFORMS,  the  International  Committee  on  Systems  Engineering  (INCOSE),  the  Society  for  Decision 
Professionals,  and  the  Lean  Systems  Society. 

Dr.  Parnell  is  currently  a  visiting  professor  of  industrial  engineering  at  the  University  of  Arkansas,  where  he  teaches  deci¬ 
sion  analysis,  systems  engineering,  and  project  management.  He  is  also  a  senior  executive  principal  at  Innovative  Deci¬ 
sions  Inc. 


29th  MORS  President:  Brian  McEnany  (1994-1995) 

Mr.  Brian  R.  McEnany  was  elected  to  the  Board  of  Directors  in  1990.  He  subsequently  chaired 
two  working  groups,  became  Vice  President  for  Professional  Affairs  in  1993,  and  served  as 
President  of  MORS  from  1994  to  1995.  In  1999,  he  was  elected  a  Fellow  of  the  Society  (FS).  He 
has  also  served  as  a  WINFORM’s  Trustee. 

His  received  a  bachelor  of  science  degree  from  the  United  States  Military  Academy  in  1962 
and  master  of  science  degrees  in  ORSA/statistics  and  in  management  science  from  Rensselaer 
Polytechnic  Institute  in  January  1970. 


After  serving  22  years  in  the  US  Army,  he  retired  as  a  Lt.  Colonel  in  1984  and  joined  Science  Applications  International 
Corporation  (SAIC).  Much  of  his  career  with  SAIC  was  devoted  to  applying  decision  analytic  and  modeling  support  in 
finding  solutions  to  US  Army  and  US  Marine  Corps  study  requirements.  As  an  Assistant  Vice  President  for  Technology, 
he  managed  and  developed  theater  models  for  the  Joint  Staff  and  helped  convert  military  tactical  knowledge  into  com¬ 
bat  instruction  sets  for  computer-generated  forces  for  the  Army’s  Close  Combat  Tactical  Trainer,  UK  Combined  Arms 
Tactical  Trainer,  and  STRICCM’s  Dismounted  Warrior  Network.  Cther  projects  included  the  development  of  methodolo¬ 
gies  to  assess  and  measure  progress  in  humanitarian  de-mining  programs  for  the  Department  of  State. 


Mr.  McEnany  continues  to  provide  advice  and  assistance  to  the  leadership  of  MCRS  and  intermittently  reviews  publica¬ 
tions  and  papers  as  an  associate  editor  of  the  Military  Operations  Research  journal.  Now  retired,  he  is  writing  articles 
and  a  book  about  West  Point  during  the  Civil  War. 

30th  MORS  President:  Christine  Fossett  (1995-1996) 

Ms.  Christine  Fossett  served  as  Vice  President  for  Administration  in  1993-1994,  and  Vice 
President  of  Professional  Affairs  in  1994-1995  prior  to  being  elected  President  in  1995.  She  was 
elected  Fellow  of  the  Society  in  1998. 

Ms.  Fossett  attended  Purdue  University  for  her  undergraduate  studies  and  received  her  mas¬ 
ter’s  degree  in  medical  sociology  with  a  minor  in  public  health  from  the  University  of  Pittsburgh 
in  1972. 


In  her  early  career,  Ms.  Fossett  worked  at  the  Pittsburgh  Board  of  Public  Education  evaluating 
federally  funded  programs;  the  Harrisburg,  Pennsylvania,  Governor’s  Justice  Commission  evaluating  and  planning  for 
the  use  of  federal  Law  Enforcement  Alliance  of  America  (LEAA)  money;  and  for  the  Washington,  DC,  Housing  and  Urban 
Development  (HUD)  Community  Block  grant  programs.  She  joined  the  Government  Accountability  Gffice  (GAG,  formerly 
General  Accounting  Office)  in  1980. 


At  GAO,  Ms.  Fossett  worked  on  numerous  programs,  including  assessing  DOD’s  Joint  Test  and  Evaluation  program  and 
evaluating  the  credibility  of  selected  weapon  systems  through  operational  effectiveness  simulations;  assessing  data 
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and  analyses  used  to  support  requirements  for  electronic  warfare  systems;  reviewing  Trident  and  START  arms  control 
and  force  structure  issues  as  part  of  a  broader  review  of  strategic  TRIAD;  evaluating  antiterrorism  measures  for  selected 
infrastructure  components  (mass  transportation  and  federal  courts)— this  followed  the  Beirut  bombing;  and  synthesizing 
evaluations  of  the  impact  of  the  Special  Supplemental  Food  Program  for  Women,  Infants,  and  Children  (WIC). 


Ms.  Fossett  also  worked  in  GAO’s  Defense  Capabilities  and  Management  (DCM)  (formerly  National  Security  International 
Affairs  Division  [NSIAD])  working  on  recruiting  and  attrition  issues,  especially  regarding  criminal  backgrounds.  This  work 
resulted  in  more  thorough  screening  for  criminal  conduct. 


Ms.  Fossett  retired  from  GAO  at  the  end  of  2007  and  continues  to  provide  advice  to  MORS  leadership.  ® 

A  TRIBUTE  TO 

MR.  SEYMOUR 

Jim  Bexfield,  FS,  jim_bexfield@comcast.net 

Philip  Major,  IDA,  pmajor@ida.org;  and  Bob  Sheldon,  FS,  bs@group-w-inc.com 


r.  Seymour  (Sy)  J. 
Deitchman,  the  MORS 
Vance  R.  Wanner  Me¬ 
morial  Award  laureate 
in  2000,  died  on  Octo¬ 
ber  11, 2013.  Sy  was 
interviewed  for  the  MORS  oral  history 
program  on  September  12,  2008  and 
October  8,  2008  and  the  interview  was 
published  in  Military  Operations  Re¬ 
search  (Volume  15,  Number  2,  2010). 

Mr.  Deitchman  served  in  private 
industry,  in  the  Defense  Advanced  Re¬ 
search  Projects  Agency  (DARPA)  and 
the  Office  of  the  Secretary  of  Defense 
(OSD),  as  vice  president  for  programs 
at  the  Institute  for  Defense  Analyses 
(IDA),  and  as  a  member  of  US  Gov¬ 
ernment  and  North  Atlantic  Treaty 
Organization  (NATO)  advisory  panels. 
Since  1982,  he  served  as  a  member 
and  then  as  a  Consultant  and  Special 
Advisor  to  the  Naval  Studies  Board 
of  the  National  Research  Council.  Dur¬ 
ing  his  career,  he  conducted  detailed 
technical  analyses,  managed  research 
and  development  programs,  was  a 
corporate  officer,  and  wrote  six  pro¬ 
fessional  books,  including  On  Being  a 
Superpower:  And  Not  Knowing  What 


to  Do  about  it,  published  in  1999,  as 
well  as  more  than  50  published  pa¬ 
pers  and  reports. 


Sy  spent  30  years  at  IDA,  interrupted 
by  two  leaves  of  absence— totaling 
five  years— to  serve  in  the  Depart¬ 


ment  of  Defense.  Sy  joined  IDA  as  a 
research  staff  member  in  1960  from 
the  Cornell  Aeronautical  Labora¬ 
tory.  From  1969  to  1988,  Sy  reported 
directly  to  the  IDA  President,  helping 
oversee  the  quality  and  effectiveness 
of  IDA’S  research  program,  first  as 
a  Director  and  then  as  an  Assistant 
Vice  President  and  Vice  President. 
Over  that  period,  he  made  many 


contributions  to  both  the  substance 
of  IDA’S  research  for  the  govern¬ 
ment  and  the  management  of  the 
Institute.  Sy’s  substantive  contribu¬ 
tions  spanned  an  enormous  range  of 
research  topics,  including  counter¬ 
insurgency  technologies  and  opera¬ 
tions,  tactical  warfare  modeling  and 
simulation,  air  combat  testing,  bal¬ 
listic  missile  defense,  naval  warfare, 
and  manned  space  flight,  to  name 
just  a  few.  Perhaps  most  noteworthy 
in  terms  of  research  management 
was  the  essential  role  Sy  played  in 
reorganizing,  refocusing,  and  rein¬ 
vigorating  IDA  in  the  1980s  under 
the  leadership  of  then  IDA  President 
General  Andrew  Goodpaster,  USA 
(ret).  During  that  time,  Sy  assumed 
line  management  responsibilities  for 
most  IDA  research  activities,  helping 
create  a  culture  of  responsiveness 
to  sponsor  needs  and  establishing 
new  IDA  research  groups  focused  on 
emerging  DoD  priorities  in  informa¬ 
tion  technology,  cost  analyses,  and 
operational  testing.  Sy’s  intelligence, 
common  sense,  and  dedication  to 
national  security  were  demonstrated 
daily  over  a  long  and  successful 
career  at  IDA. 
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Sy  was  one  of  the  nation’s  most 
accomplished  and  well-regarded 
analysts  of  defense  issues.  He  was  re¬ 
nowned  for  his  ability  to  integrate  the 
multiple  dimensions  of  problems  into 
a  form  that  was  readable,  understand¬ 
able,  and  actionable.  In  doing  so,  he 


exemplified  the  highest  standards  of 
operations  research,  which  include  a 
passion  for  clear  thinking,  creative  op¬ 
tion  development,  objective  assess¬ 
ment,  sound  analysis,  integrity,  and 
depiction  of  conclusions  in  a  form  that 
policymakers  can  use. 


In  recent  years,  Sy  was  engaged 
with  the  MORS  Social  Science  Com¬ 
munity  of  Practice.  He  shared  les¬ 
sons  from  the  use  of  social  science 
during  the  Vietnam  War  that  are 
relevant  to  current  studies  of  irregu¬ 
lar  warfare.  ® 


Golf  Queuing 


^  PuzzLOR 
D^ision 

^  engineering 
\  research  fronf'/irginu 


■he  creation  of  John 


lonal  Excellence  group.  Hee^m 
^^sxel  University  ( 1 996)  andWi 
MmmtiLCommonwealth  Univen 


Golfing  can  be  an  enjoyable  and  rewarding  way  to  spend  your  time.  Despite  the  attraction  and  fun  of  the  game  there 
can  be  many  challenges.  One  of  the  more  common  challenges  for  experienced  players  is  waiting  for  slower  players  to 
before  the  experienced  player  can  start. 


Question:  How  much  time 
on  average  (in  minutes)  will 
a  player  save  if  you  convert 
to  the  Priority  queuing  from 
First-In-First-Out  queuing? 


As  the  owner  of  a  9  hole  golf  course,  you  currently  have 
a  First-In-First-Out  policy.  In  other  words,  faster  players 
are  not  allowed  to  Jump  ahead  of  slower  players.  You  are 
considering  changing  this  First-In-First-Out  policy  to  a  Prior¬ 
ity  queuing  policy  to  allow  faster  players  to  jump  ahead  of 
slower  players  in  between  holes. 

Players  arrive  at  your  golf  course  at  an  interarrival  time  of  1 0  minutes, 
exponentially  distributed.  The  players  on  your  golf  course  have  3  different 
skill  levels.  Fast  players  complete  holes  at  an  average  of  5  minutes.  Medium 
players  complete  holes  at  an  average  of  7  minutes.  Slow  players 
complete  holes  at  an  average  of  10  minutes.  All  distributions  are 
normal  and  have  a  standard  deviation  of  1  minute.  Player  skill 
level  is  randomly  distributed  (1/3  Fast,  1/3  Medium,  1/3  Slow). 


Assume  players  start  golfing  as  soon  as  they  arrive  on 
the  course  and  that  the  system  has  achieved  steady 
state.  Each  player  is  golfing  individually  (not  in  a 
group)  and  players  must  go  in  sequential  order 
from  hole  1  to  hole  9.  Players  can  only 
jump  the  queue  if  a  slower  player  has 
not  yet  started  the  hole. 


Send  your  answer  to  puzzlor@ 
gmail.com  by  April  1 5th,  2014. 
The  winner,  chosen  randomly 
from  correct  answers,  will 
receive  a  $25  Amazon  Gift 
Card. 


John  is  th^Sr.  Director  of  ^ 


Corporation  in  the 
gamed  his  BSc.  in  chemical 
’gis  MSciiifi^op^ations, 
ty(2005).  o 
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“™“1\/li  estones 


George  Akst,  MORS  USMC  Sponsor,  is  now  the  head  of 
the  75-person  Marine  Corps  Analysis  Division. 

Nathan  Bastian,  last  year’s  Bonder  scholarship  recipient, 
will  chair  a  session  at  INFORMS  in  2014. 

Aaron  Burciaga  recently  left  the  service  and  is  a  civilian 
OR  analyst.  He  is  serving  both  on  the  MAS  council  and 
the  MORS  Board  of  Directors.  Aaron  was  recently  elected 
Vice  President  of  the  Maryland  Chapter  of  INFORMS.  This 
position  automatically  progresses  to  the  role  of  President 
in  2015. 

Major  (USMC)  David  Cote  has  started  the  Summit  Project, 
a  living  memorial  that  pays  tribute  to  fallen  service  mem¬ 
bers  from  Maine  who  died  during  the  wars  in  Afghanistan 
and  Iraq.  For  information,  visit  http://mainememorial.files. 
wordpress.com/2013/09/the-summit-project-brochure.pdf 

MAS  Secretary/Treasurer  Walt  DeGrange  is  stepping 
down  after  many  years  of  service  in  these  positions  as  well 
as  retiring  from  active  naval  service.  We  wish  Commander 
DeGrange  every  success  in  the  future! 

Tom  Denesia  completed  the  an¬ 
nual  Pike’s  Peak  Ascent,  improv¬ 
ing  his  time  from  last  year  and  fin¬ 
ishing  in  5  hours,  46  minutes,  and 
54  seconds.  Tom  admits  that  this 
may  seem  like  a  long  time  for  a 
half  marathon,  but  he  reminds  us 
that  the  event  starts  at  6,285  feet  and  ends  up  at  14,100 
feet  above  sea  level.  He  claims  to  have  jogged  and  walked 
fast,  even  though  breathing  above  the  10,000-foot  level  is 
a  real  challenge! 

Jeff  Hyink  was  promoted  to  Captain,  USN,  and  has  as¬ 
sumed  duties  as  the  Operations  Analysis  Program  Officer 
at  the  Naval  Postgraduate  School.  Jeff’s  previous  assign¬ 
ment  was  as  Sea  Strike  branch  head  at  OPNAV  N81 . 


the  Analytical  Hierarchy  Process  Using  Discrete  Dynamical 
Systems  in  Decision  Analysis.”  The  award  recognizes  the 
best  paper  in  computational  methods. 


MORS  President-elect  Raphael 
Matos  recently  completed  his 
PhD  at  Walden  University. 


Norm  Reitter,  MORS  Direc¬ 
tor,  has  been  named  Director, 
Analytics  at  CANA  Advisors,  a 
small,  woman-owned,  veteran- 
owned  business,  based  in 
Northern  Virginia  that  provides 
integrated  logistics,  supply 
chain,  and  company  acquisition  services  to  military  and 
commercial  clients. 

Papers  from  MAS  former  president  Patrick  Driscoll,  cur¬ 
rent  MAS  Vice-President  Chris  Arney,  former  Steinhardt 
recipient  Don  Gaver,  and  others  appear  in  a  special  issue 
of  the  International  Journal  of  Operations  Research  and 
Systems  Science,  edited  by  MAS  President  Dr.  William  P. 
Fox.  See  the  MAS  President’s  article  for  details. 

This  could  have  happened.  Mike  Garrambone  was  mak¬ 
ing  a  quick  stop  at  the  commissary  accompanied  by  his 
mentor  and  friend  Mr.  E.B.  Vandiver,  FS.  Mike  was  a  little 
unnerved  when  asked  the  question  he  had  heard  many 
times  before,  “Paper  or  plastic?”  His  mentor  took  note  of 
his  demeanor,  which  prompted  Mike  to  say,  “Well,  why 
do  they  always  have  to  ask  that  question?”  In  a  sage 
voice,  his  mentor  quickly  replied,  “Garrambone,  I’ve  tried 
my  very  best  with  you.  Every  analyst  knows  that  baggers 
can’t  be  choosers.” 

Submit  items  for  Milestones  to  phalanx@mors.org 


MAS  President  Dr.  William  Fox  received  the  Harden- 
Simmons  prize  from  the  American  Society  for  Engineering 
Education  (ASEE)  for  his  paper,  “Mathematical  Modeling  of 
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La  s|\/o  rd 

Joint  Warfare  Analysis: 

The  Key  to  Shaping  DoD's  Future 

Arthur  H.  Barber  III,  Deputy  Director,  Assessment  Division 
(N81),  Office  of  the  CNO,  arthur.barber@navy.mil 


Affordability  will  dominate 
the  US  military’s  planning 
for  the  foreseeable  future. 
America  has  a  history  of 
reducing  defense  spend¬ 
ing  following  a  war,  and 
large-scale  operations  in  our  lon¬ 
gest  war  are  now  coming  to  a  close 
so  defense  funding  is  going  down. 
Because  of  the  debt  and  deficit  situ¬ 
ations  that  the  nation  faces  today,  it 
is  likely  that  these  developing  reduc¬ 
tions  in  defense  spending  will  endure 
for  a  longer  period  of  time  than  in 
previous  drawdowns.  The  US  military 
is  caught  between  the  funding  con¬ 
straints  of  this  national  budget  envi¬ 
ronment  and  the  steadily  increasing 
capability  demands  of  pacing  new 
and  globally  proliferating  technolo¬ 
gies  that  sharply  target  US  military 
strengths.  Focusing  flat  or  declining 
defense  resources  on  the  capabili¬ 
ties  that  matter  most  will  be  critically 
important  over  the  next  decade. 

There  is  a  great  opportunity  in  this  situ¬ 
ation  to  use  a  thoughtfully  structured 
program  of  joint  capability  analysis 
to  identify  the  most  cost-effective 
concepts  of  operations  and  types  of 
capabilities  to  deal  with  our  military 
challenges.  Such  analysis  could  also 
potentially  identify  the  most  promis¬ 
ing  directions  for  development  of  new 


capabilities  to  deal  with  new  classes  of 
threats.  Because  so  much  of  what  we 
know  about  threats  and  much  of  what 
is  most  effective  in  our  future  capability 
are  both  classified  above  the  SECRET 
level,  this  analysis  would  have  to  be 
done  in  a  highly  secure  environment. 
Because  US  military  operations  are 
conducted  jointly,  the  security  environ¬ 
ment  would  also  have  to  be  cross-ser¬ 
vice.  Despite  the  existence  of  a  frame¬ 
work  for  coordinating  joint  capability 
analysis  that  is  tri-chaired  by  Office  of 
the  Secretary  of  Defense  (OSD)  Policy, 
OSD  Cost  Analysis  and  Program 
Evaluation  (CAPE),  and  the  Joint  Staff 
J-8  but  seldom  exercised,  this  type  of 
analytic  coordination  is  not  happen¬ 
ing  today  on  the  scale  and  at  the  rate 
that  the  fiscal  situation  requires.  Such 
work  as  is  being  done  is  being  done 
separately  by  diverse  segments  of  the 
Department  of  Defense  (DoD),  each 
working  to  their  own  well-intentioned 
individual  agenda.  The  analytic  com¬ 
munity  is  not  being  utilized  efficiently 
or  effectively  on  a  DoD-wide  basis  to 
do  what  we  know  how  to  do.  No  one  is 
coordinating  our  collective  joint  efforts. 
We  need  to  do  better. 

The  foundation  of  warfare  analysis 
is  the  definition  (always  somewhat 
speculative)  of  a  campaign  or  sce¬ 
nario  that  provides  the  threat,  geogra¬ 


phy,  military  objectives  of  both  sides, 
political  environment  (what  nations 
are  involved  in  what  manner),  and  a 
projected  timeline  of  events  leading 
up  to  conflict.  This  can  be  either  an 
existing  current-year  operational  plan 
(OPLAN)  campaign  from  a  combatant 
commander  (COCOM),  or  a  future- 
year  defense  planning  scenario  from 
the  OSD  (Policy).  Both  exist  in  sig¬ 
nificant  numbers  and  the  process  of 
developing  new  or  updating  existing 
ones  is  fairly  healthy,  even  if  painfully 
slow.  A  key  issue  for  analytic  purpos¬ 
es  is  what  scenario  or  combination  of 
them  should  be  used  in  what  man¬ 
ner  to  provide  the  analytic  framework 
for  force  and  program  planning.  The 
exact  blend  of  OPI_ANs  and  future- 
year  planning  scenarios  to  be  used 
for  force  structure  analysis  is  always  a 
point  of  debate  within  DoD,  especially 


around  Quadrennial  Defense  Reviews, 
but  for  most  other  forms  of  analysis, 
each  scenario  is  evaluated  separately. 
So  the  scenario  foundation  for  analy¬ 
sis  is  fairly  diverse  and  robust.  But 
what  about  the  structure  above  it? 

The  second  step  in  joint  warfare 
analysis  is  filling  in  the  structural 
details  of  a  campaign:  how  we  project 
the  enemy  will  proceed  to  achieve 
military  objectives  and  how  we  would 
deploy  and  employ  our  own  forces 
and  capabilities  to  defeat  this  and 
achieve  our  own  objectives.  Obviously, 
this  is  even  more  speculative  than 
scenario  definition.  The  process  for 
doing  this  involves  a  significant  staff 
effort  and  the  application  of  warfighter 
military  judgment  through  workshops 
and  wargames,  either  at  the  COCOM 
level  (for  OPLANS)  or  led  by  the  Joint 
Staff  for  future-year  scenarios.  Once 
again,  this  is  a  step  where  there  is  a 
significant  amount  of  effort  under¬ 
way.  What  is  missing  from  this  step  at 
both  the  COCOM  and  the  OSD  levels 
is  the  systematic  application  of  fully 
joint,  highly  classified  campaign-level 
analysis  to  inform  the  selection  of  the 
courses  of  action  and  types  of  forces 
and  capabilities  that  are  most  likely  to 
be  successful  in  achieving  the  desired 
outcome.  Much  of  the  work  today 
uses  no  analysis  at  all;  the  rest  uses 
table-top  insights  with  spreadsheets 
or  analytic  structures  and  tools  that  do 
not  incorporate  all  services  appropri¬ 
ately  and  do  not  include  the  highest 
classification  and  most  effective  US 
capabilities.  The  services  of  the  ana¬ 
lytic  community  are  not  being  used  in 
the  way  that  they  could  and  should  be. 

So  why  is  this  critical  step  in  joint 
warfare  analysis,  the  application  of 
campaign-level  analytic  techniques, 
being  underperformed?  The  first 
reason  is  a  philosophical  prejudice 
against  this  type  of  analysis  due  to  the 
complexity  of  the  models  and  the  long 


chain  of  assumptions  that  are  used 
in  their  inputs.  OSD  CAPE  (formerly 
PA&E)  disestablished  their  unique 
capability  and  staff  for  joint  campaign 
analysis  a  few  years  ago  over  this 
issue,  as  did  US  Pacific  Command 
(PACOM).  Although  the  Joint  Staff, 
PACOM,  CENTCOM,  and  multiple 
OSD  offices  other  than  CAPE  have 
seen  a  continuing  need  for  this  type 
of  work  and  sought  to  set  up  replace¬ 
ment  capability  since  then,  none  have 
had  the  staff  resources  and/or  analytic 
expertise  to  succeed.  The  second 
reason  for  not  having  the  right  kind  of 
joint  campaign  analysis  is  administra¬ 
tive.  It  is  extraordinarily  difficult  to  get 
bureaucratic  approval  to  put  all  the 
technical  details  for  highly  classified 
programs  from  all  services,  along  with 
the  highest-classification  threat  infor¬ 
mation,  simultaneously  on  the  same 
set  of  computers  on  a  sustained  basis 
and  then  clear  the  number  of  working- 
level  analysts  that  would  be  required 
to  do  wide-scale  joint  analysis  into 
this  whole  set  of  information. 

OSD  (CAPE)  is  correct  in  saying  that 
campaign  analysis  is  built  on  many 
debatable  assumptions  and  complex, 
labor-intensive  models.  The  power  of 
this  type  of  analysis,  however,  is  that 
it  provides  a  structured  common  joint 
warfare  framework  within  which  es¬ 
sential  elements  of  warfighting  can  be 
accounted  for  systematically.  Good 
analysts  can  use  this  framework  to 
establish  a  common  operational  con¬ 
text  for  detailed  analysis  of  specific 
issues  with  mission-level  models.  Or 
they  can  use  campaign-level  models 
to  compare  the  impact  of  changing 
scenario  assumptions  across  a  range 
of  realistic  possibilities.  The  campaign 
analysis  provides  a  frame  of  refer¬ 
ence,  underpinned  by  real  effective¬ 
ness  calculations  rather  than  purely 
military  judgment,  within  which  the 
value  of  specific  systems,  elements 
of  force  structure  and  their  arrival 


rate,  new  technology  options,  and 
various  operational  concepts  can  be 
compared  quantitatively.  The  “scores” 
that  are  the  direct  campaign  model 
outputs  are  not  the  value;  qualita¬ 
tive  comparative  insights— often  not 
obvious  from  intuition  alone— about 
what  is  likely  to  work  better  and  why 
are  the  key  and  unique  result  from  the 
rigor  of  this  framework. 

Navy  and  Air  Force  use  campaign 
analysis  extensively  for  just  these 
reasons,  and  began  teaming  to  con¬ 
duct  multiservice  campaign  analysis 
incorporating  their  Air-Sea  Battle 
concept  of  operations  in  2010  after 
OSD  disestablished  the  fully  joint 
system.  The  insights  that  come  from 
campaign-level  analysis  continue  to 
be  extremely  useful  to  the  leadership 
of  these  two  services,  and  the  prod¬ 
ucts  that  have  resulted  from  their  joint 
effort  have  been  eagerly  sought  by  a 
range  of  offices  in  OSD  and  several 
COCOMs.  The  Defense  Department 
needs  this  kind  of  joint  warfare  analy¬ 
sis  work  as  one  of  the  pieces  of  an 
analytic  foundation  for  developing  the 
best  possible  current-year  OPLANs 
and  for  cost-effectively  shaping  the 
future  US  military. 

The  Navy-Air  Force  work,  for  all  its 
strengths,  still  has  one  weakness.  Nei¬ 
ther  service  can  populate  its  respective 
campaign-analysis  computer  systems 
(which  both  run  the  same  model, 
STORM,  using  the  same  starting  data¬ 
base)  with  the  most  highly  classified  or 
“black”  programs  of  the  other  service. 
So  each  service  has  to  complete  the 
joint  campaign  with  acknowledged 
programs  then  go  off  and  do  addi¬ 
tional  runs,  separately  by  service,  to 
fully  incorporate  their  other  programs. 
Although  the  Joint  Staff  has,  after  years 
of  effort,  achieved  the  bureaucratic 
authority  to  run  campaign-analysis 
computers  with  all  classified  programs 
of  all  services,  they  do  not  have  the 


staff  capacity  or  tour  length  for  their 
largely  military  staff  to  do  sustained 
work  at  the  scale  needed  for  fully  joint 
large-scale  campaigns.  Interestingly 
enough,  various  offices  in  OSD  other 
than  CAPE  have  found  such  highly 
classified  work  useful  to  their  mission 
and  have  been  granted  approval  to  do 
it  on  an  episodic  basis  using  nongov¬ 
ernment  contractor  facilities  and  staff. 
There  is  a  clear  need  for  a  standing  fully 
joint  government-run  campaign-level 
analytic  process  with  full  service  partici¬ 
pation.  No  one  has  taken  charge  at  the 
joint  level  to  assemble  the  authorities 
and  resources  to  make  this  happen. 

I  spend  a  great  deal  of  time  trying  to 
stay  aware  of  what  analytic  work  is 
being  done  throughout  the  Defense 
Department  on  warfighting  capability 
issues  of  interest  to  the  Navy.  This  is 
not  easy;  each  time  I  think  that  I  have 
found  it  all,  I  discover  new  pockets  of 
well-intentioned  effort  being  per¬ 
formed  somewhere,  much  of  it  being 
done  by  the  technical  community  of 
federally  funded  and  university  R&D 
centers  with  money  from  one  separate 
office  or  another  in  OSD.  Most  of  it 
is  good  in  technical  quality,  but  the 
work  in  each  place  is  often  based  on 
entirely  different  starting  assumptions 
about  concepts  of  operations,  sce¬ 
nario,  etc.,  than  other  work  on  related 
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Union,  the  once  bipolar  world  would 
be  replaced  by  a  multipolar  one.  The 
emerging  national  and  international 
security  problems  would  likely  be  quite 
different  from  those  experienced  dur¬ 
ing  the  Cold  War.  We  would  be  faced 
not  only  with  new  problems,  but  also 
with  new  kinds  of  problems  requiring 
new  tools,  new  ideas,  and  new  analyt¬ 
ic  approaches  for  solution.  A  different 
format  would  be  needed  for  the  proper 
exchanges  among  analysts.  Thus,  the 
Cornwallis  Group  experimented  with 
scheduling  fewer  papers  and  allow¬ 


subjects  done  elsewhere.  This  makes 
for  good  debates  about  assump¬ 
tions  versus  analytic  conclusions, 
but  makes  it  very  difficult  for  any 
senior  leader  to  integrate  the  results 
into  a  coherent  picture,  even  if  they 
were  aware  of  all  the  results  and  the 
divergent  assumptions  behind  them. 
And  there  is  significant  inadvertent 
redundancy  of  effort  simply  due  to 
lack  of  awareness  of  what  others  are 
doing  or  have  done.  There  is  no  single 
place  or  forum  for  coordination  or 
even  exchange  of  information  of  who 
is  working  on  what  analytic  task.  Each 
service  has  an  internal  requirement  for 
such  sharing  and  coordination  within 
their  own  service,  but  there  is  no 
requirement  or  method  for  exchang¬ 
ing  or  sharing  such  information  within 
OSD  or  at  the  joint  level. 

When  resources  decline,  the  impor¬ 
tance  of  analysis  increases.  When 
every  dollar  in  DoD  has  to  be  used 
efficiently,  a  carefully  structured  and 
comprehensive  but  nonredundant 
program  for  coordinated  joint  analysis 
should  be  a  key  element  of  making 
this  happen.  The  Defense  Department 
is  in  fiscal  extremis  right  now,  and  we 
do  not  have  in  place  a  structured  joint 
program  of  analysis  that  operates  at 
the  scale  or  with  the  focus  needed  to 
support  DoD  leadership’s  ability  to 


ing  presenters  sufficient  time  to  fully 
explore  their  ideas  and  approaches  in 
contradiction  to  the  traditional  format 
of  symposia.  The  traditional  formula 
provides  for  the  maximum  number 
of  presentations,  with  20  minutes  of 
presentation  and  a  few  minutes  for 
questions.  The  Cornwallis  format  al¬ 
lows  for  a  great  interchange  between 
the  speakers  and  the  audiences.  This 
unique  approach  has  worked  success¬ 
fully,  as  the  bookshelf  of  proceedings 
for  the  first  1 8  symposia  attest.  The 
proceedings  are  filed  electronically  on 


make  good  capability-based  resource 
decisions.  Each  service  separately 
has  appropriately  structured  analytic 
programs  that  its  own  leaders  use 
internally,  and  the  services  sometimes 
collaborate  where  they  see  opportuni¬ 
ties,  but  they  are  doing  this  indepen¬ 
dently  as  coalitions  of  the  willing.  OSD 
operates  on  the  separate  analytic 
agendas  of  its  multiple  organizations. 
The  DoD-wide  joint  analytic  process 
has  actually  gone  backward  over  the 
last  several  years,  with  the  OSD/J-8 
chaired  joint  analytic  steering  commit¬ 
tee  falling  into  disuse  and  DoD-wide 
campaign  analysis  being  abandoned. 
Big  program  and  force  structure  deci¬ 
sions  are  being  made  too  often  on  the 
basis  of  individual  topical  and  nonjoint 
analysis,  if  analysis  is  used  at  all.  We 
can  and  must  do  better  than  this. 
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the  Cornwallis  website  (www.thecorn- 
wallisgroup.org). 

As  noted  earlier,  I  am  proud  to  be 
invited  to  deliver  the  Professor  Ronnie 
Shephard  Memorial  Address  at  the 
traditional  Thursday  banquet  during 
the  symposium. 

Further  information  can  be  found 
on  the  ISMOR  website,  www.ismor. 
com,  or  by  contacting  Gene  Visco 
at  evisco4@cfl.rr.com  or  Eugene. 
visco@lmco.com.  ® 
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